Interactive user interface functionality for lighting devices or system

ABSTRACT

An example of a lighting system includes intelligent lighting devices, each of which includes a light source, a communication interface and a processor coupled to control the light source. In such a system, at least one of the lighting devices includes a user input sensor to detect user activity related to user inputs without requiring physical contact of the user; and at least one of the lighting devices includes an output component to provide information output to the user. One or more of the processors in the intelligent lighting devices are further configured to process user inputs detected by the user input sensor, control lighting and control output to a user via the output component so as to implement an interactive user interface for the system, for example, to facilitate user control of lighting operations of the system and/or to act as a user interface portal for other services.

RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.16/366,436, Filed Mar. 27, 2019, entitled “INTERACTIVE USER INTERFACEFUNCTIONALITY FOR LIGHTING DEVICES OR SYSTEM,” the disclosure of whichis entirely incorporated herein by reference.

U.S. application Ser. No. 16/366,436 is a continuation of U.S.application Ser. No. 15/720,546, Filed Sep. 29, 2017, now U.S. Pat. No.10,278,261, Issued Apr. 30, 2019, entitled “INTERACTIVE USER INTERFACEFUNCTIONALITY FOR LIGHTING DEVICES OR SYSTEM,” the disclosure of whichis entirely incorporated herein by reference.

U.S. application Ser. No. 15/720,546 is a continuation of U.S.application Ser. No. 15/259,612, Filed Sep. 8, 2016, now U.S. Pat. No.9,781,811, Issued Oct. 3, 2017 entitled “INTERACTIVE USER INTERFACEFUNCTIONALITY FOR LIGHTING DEVICES OR SYSTEM,” the disclosure of whichis entirely incorporated herein by reference.

U.S. application Ser. No. 15/259,612 is a continuation of U.S.application Ser. No. 13/971,428, Filed Aug. 20, 2013, now U.S. Pat. No.9,462,663, Issued Oct. 4, 2016 entitled “INTERACTIVE USER INTERFACEFUNCTIONALITY FOR LIGHTING DEVICES OR SYSTEM,” the disclosure of whichis entirely incorporated herein by reference.

U.S. application Ser. No. 13/971,428 is a continuation-in-part of U.S.application Ser. No. 13/903,428, Filed May 28, 2013, now U.S. Pat. No.9,504,132, Issued Nov. 22, 2016 entitled “DISTRIBUTED PROCESSING USINGRESOURCES OF INTELLIGENT LIGHTING ELEMENTS OF A LIGHTING SYSTEM,” thedisclosure of which is entirely incorporated herein by reference.

U.S. application Ser. No. 13/971,428 is also related to U.S. applicationSer. No. 13/903,330, Filed May 28, 2013 entitled “LIGHTING NETWORK WITHAUTONOMOUS COMMISSIONING,” the disclosure of which is entirelyincorporated herein by reference.

U.S. application Ser. No. 13/971,428 is also related to U.S. applicationSer. No. 13/964,564, Filed Aug. 12, 2013 entitled “LIGHTINGELEMENT-CENTRIC NETWORK OF NETWORKS,” the disclosure of which isentirely incorporated herein by reference.

TECHNICAL FIELD

The present subject matter relates to techniques and equipment toprovide an interactive user interface for lighting purposes, forexample, to allow a user to control lighting and/or to access otherservices via a lighting device and associated data network.

BACKGROUND

Electrical lighting has become commonplace in modern society. Electricallighting devices are commonly deployed, for example, in homes, buildingsof commercial and other enterprise establishments, as well as in variousoutdoor settings. Even in a relatively small state or country, there maybe millions of lighting devices in use.

Traditional lighting devices have tended to be relatively dumb, in thatthey can be turned ON and OFF, and in some cases may be dimmed, usuallyin response to user activation of a relatively simple input device.Lighting devices have also been controlled in response to ambient lightdetectors that turn on a light only when ambient light is at or below athreshold (e.g. as the sun goes down) and in response to occupancysensors (e.g. to turn on light when a room is occupied and to turn thelight off when the room is no longer occupied for some period). Oftentraditional lighting devices are controlled individually or asrelatively small groups at separate locations.

With the advent of modern electronics has come advancement, includingadvances in the types of light sources as well as advancements innetworking and control capabilities of the lighting devices. Forexample, solid state sources are now becoming a commercially viablealternative to traditional light sources such as incandescent andfluorescent lamps. By nature, solid state light sources such as lightemitting diodes (LEDs) are easily controlled by electronic logiccircuits or processors. Electronic controls have also been developed forother types of light sources. As increased processing capacity finds itsway into the lighting devices, it becomes relatively easy to incorporateassociated communications capabilities, e.g. to allow lighting devicesto communicate with system control elements and/or with each other. Inthis way, advanced electronics in the lighting devices as well as theassociated control elements have facilitated more sophisticated lightingcontrol algorithms as well as increased networking of lighting devices.

However, there have also been proposals to further enhance lightingcontrols. For example, it has been proposed that a lighting device mayinclude a sensor and processing capability to detect gestural inputsfrom a user. If the sensor detects touch, the user must approach thedevice or an associated control panel and contact the touch sensor in anappropriate manner to input a gestural corresponding to the user'sdesired control of the light. More recent developments in gesturalsensing technologies eliminate the need for actual touching, but suchdevices still typically require that the user make the appropriategesture in fairly close proximity to the sensor on the device or at thecontrol panel.

There have also been efforts to develop speech-command responsivecontrol of lighting, using advanced speech recognition technologies.

In a somewhat related field a variety of entities are proposing controlsfor lighting and other functions in a building from a variety ofportable user devices, for example, from remote controls or from mobiledevices such as smartphones or tablet computers.

Despite such recent efforts, there is still room for further improvementin the user interface with a lighting system and/or in the functionsthat a lighting system may offer through its user interface.

BRIEF DESCRIPTION OF THE DRAWINGS

The drawing figures depict one or more implementations in accord withthe present concepts, by way of example only, not by way of limitations.In the figures, like reference numerals refer to the same or similarelements.

FIG. 1A is a functional block diagram of a simple example of a systemhaving intelligent lighting devices, at least some of which includecomponents and are configured to implement an interactive userinterface.

FIG. 1B is a functional block diagram of an example of an intelligentlighting device that may be used in the system of FIG. 1A.

FIG. 2 is a flow chart of a simple example of a procedure fordistributed processing, involving resource sharing, which may beimplemented in a lighting system like that of FIG. 1 as part of animplementation of an interactive user interface.

FIG. 3 is an alternative diagram of selected aspects of the system ofFIG. 1, representing an example of multiple-instance server type ofdistributed processing.

FIG. 4 is a simplified process flow diagram illustrating an example ofan interactive user interface procedure, where the interactive userinterface is provided via the intelligent lighting devices.

FIG. 5 is signal flow diagram providing an alternative form ofillustration of two examples of process flows for the interactive userinterface provided via the intelligent lighting devices.

FIG. 6 is a flow chart another example of an interactive user interfaceprocedure.

FIG. 7 is a is a simplified functional block diagram of a computer thatmay be configured as a host or server, for example, to function as theexternal server or a server if provided at the premises in the system ofFIG. 1A.

FIG. 8 is a simplified functional block diagram of a personal computeror other user terminal device, which may be used as the remote accessterminal, in the system of FIG. 1A.

FIG. 9 is a simplified functional block diagram of a mobile device, asan alternate example of a user terminal device, for possiblecommunication in or with the system of FIG. 1A.

DETAILED DESCRIPTION

In the following detailed description, numerous specific details are setforth by way of examples in order to provide a thorough understanding ofthe relevant teachings. However, it should be apparent to those skilledin the art that the present teachings may be practiced without suchdetails. In other instances, well known methods, procedures, components,and/or circuitry have been described at a relatively high-level, withoutdetail, in order to avoid unnecessarily obscuring aspects of the presentteachings.

As lighting devices incorporate more intelligence, people are beginningto add more functionality, such as more sophisticated userinteractivity. The world is becoming interconnected. The trend intechnologies that control lighting is toward an “Internet of things” inwhich more and more machines are interconnected to communicate with eachother and interact with the users via the Internet. However, there aremany diverse ways to access the Internet, for example, with a computervia wired or fiber network (even with a WiFi local link) or with amobile device (e.g. smartphone or tablet) via any of the variousavailable public and private wireless networks.

For lighting, the lighting devices and controllers and possibly somecentral control element (e.g. a server) may communicate with each othervia a network. The user in turn communicates with such a system via theInternet using one of these common access techniques. So, the user iscoming in from another network that may be separate from the networkingused for communications of the lighting system elements. The user alsohas their own device, albeit of their choosing, but separate and inaddition to the elements of the lighting system. Such user access may bepart of the problem. For example, use of other access technologies addsto the complexity of the system; and the integration of the lightingnetwork with other user devices, may entail use of separate user deviceprogramming in addition to special programming in the lighting system,and/or may increase overall costs. In some cases, the additional devicesand/or their software may not be adequately adapted to the lightingsystem and its operations.

To improve the user experience and provide a more effective or moreefficient user interface, the various examples of a lighting systemdiscussed below and shown in the drawings offer an interactive userinterface implemented with the input and/or output components andassociated processing functionality in one or more of the lightingdevices. Stated another way, the lighting devices themselves implementthe interactive user interface to the lighting system, and the userinteracts with the lighting system via the lighting devices.

The system having the user interface via the lighting devices also willenable use of the light fixture or lamp as a user interface portal, foraccessing the system itself for lighting functions and possibly othernon-lighting functions controlled via the network. However, the systemalso offers the user access to outside network resources, e.g. via theInternet. The fixture may offer a voice interface (recognition for inputand audible sound/speech output) or a more visual interface (e.g.movement or gesture recognition and a projector or the like for visibleoutput) or various combinations of audio and visual input/outputcapabilities. The light outputs may also be controlled in a manner thatconveys information to a person in the vicinity. The user no longerneeds to operate a separate terminal, such as their computer, tablet orsmartphone, although other devices may utilize the network either foradditional user interfaces or for control or communications via thelighting system and the network facilities of the lighting system.

Reference now is made in detail to the examples illustrated in theaccompanying drawings and discussed below. FIG. 1A illustrates anexample of the system 10 in block diagram form. The illustrated exampleof the system 10 includes a number of intelligent lighting devices 11,such as fixtures or lamps or other types of luminaires. Severaldifferent configurations of the lighting devices 11 are shown by way ofexamples. The represented differences amongst the examples of devices 11will be discussed more fully later.

The term “lighting device” as used herein is intended to encompassessentially any type of device that processes power to generate light,for example, for illumination of a space intended for use of oroccupancy or observation, typically by a living organism that can takeadvantage of or be affected in some desired manner by the light emittedfrom the device. However, a lighting device 11 may provide light for useby automated equipment, such as sensors/monitors, robots, etc. that mayoccupy or observe the illuminated space, instead of or in addition tolight provided for an organism. A lighting device 11, for example, maytake the form of a lamp, light fixture or other luminaire thatincorporates a source, where the source by itself contains nointelligence or communication capability (e.g. LEDs or the like, or lamp(“regular light bulbs”) of any suitable type). Alternatively, a fixtureor luminaire may be relatively dumb but include a source device (e.g. a“light bulb”) that incorporates the intelligence and communicationcapabilities discussed herein. In most examples, the lighting device(s)11 illuminate a service area to a level useful for a human in or passingthrough the space, e.g. regular illumination of a room or corridor in abuilding or of an outdoor space such as a street, sidewalk, parking lotor performance venue. However, it is also possible that one or morelighting devices 11 in or on a particular premises 21 served by a system10 have other lighting purposes, such as signage for an entrance or toindicate an exit. Of course, the lighting devices 11 may be configuredfor still other purposes, e.g. to benefit human or non-human organismsor to repel or even impair certain organisms or individuals.

Each respective intelligent lighting device 11 includes a light source13, a communication interface 15 and a processor 17 coupled to controlthe light source 13. The light sources may be virtually any type oflight source suitable to providing illumination that may beelectronically controlled. The light may be of the same general type inall of the lighting devices, e.g. all formed by some number of lightemitting diodes (LEDs); although in many installations, some number ofthe lighting devices 11 may have different types of light sources 13.

The processor 17 also is coupled to communicate via the interface 15 andthe network link with one or more others of the intelligent lightingdevices 11 and is configured to control operations of at least therespective lighting device 11. The processor may be implemented viahardwired logic circuitry, but in the examples, the processor 17 is aprogrammable processor such as a the central processing unit (CPU) of amicrocontroller or a microprocessor. Hence, in the example of FIG. 1A,each lighting device 11 also includes a memory 19, storing programmingfor execution by the processor 17 and data that is available to beprocessed or has been processed by the processor 17. The processors andmemories in the lighting devices may be substantially the samethroughout the devices 11 throughout the premises, or different devices11 may have different processors 17 and/or different amounts of memory19, depending on differences in intended or expected processing needs.

In the examples, the intelligence (e.g. processor 17 and memory 19) andthe communications interface(s) 15 are shown as integrated with theother elements of the lighting device or attached to the fixture orother element that incorporates the light source. However, for someinstallations, the light source may be attached in such a way that thereis some separation between the fixture or other element thatincorporates the electronic components that provide the intelligence andcommunication capabilities. For example, the communication component(s)and possibly the processor and memory (the ‘brain’) may be elements of aseparate device or component coupled and/or collocated with the lightsource 13.

In our example, the system 10 is installed at a premises 21. The system10 also includes a data communication network 23 that interconnects thelinks to/from the communication interfaces 15 of the lighting devices11, so as to provide data communications amongst the intelligentlighting devices 11. Such a data communication network 23 also isconfigured to provide data communications for at least some of theintelligent lighting devices 11 via a data network 25 outside thepremises, shown by way of example as a wide area network (WAN), so as toallow devices 11 or other elements/equipment at the premises 21 tocommunicate with outside devices such as the server/host computer 27 andthe user terminal device 29. The wider area network 25 outside thepremises, may be an intranet or the Internet, for example.

The premises 21 may be any location or locations serviced for lightingand other purposes by a networked intelligent lighting system of thetype described herein. The lighting devices 11 are located to providelighting service in various areas in or about the premises 21. Most ofthe examples discussed below focus on building installations, forconvenience, although the system may be readily adapted to outdoorlighting. Hence, the example of system 10 provides lighting and possiblyother services in a number of service areas in or associated with abuilding, such as various rooms, hallways, corridors or storage areas ofa building and an outdoor area associated with a building. Any buildingforming or at the premises 21, for example, may be an individual ormulti-resident dwelling or may provide space for one or more enterprisesand/or any combination of residential and enterprise facilities.

The lighting devices 11, as well as any other equipment of the system orthat uses the network 23 in the service areas of the premises 21,connect together with and through the network links and any other mediaforming the communication network 23. For lighting operations, thelighting devices 11 (and other system elements if any) for a givenservice area are coupled together for network communication with eachother through data communication media to form a portion of a physicaldata communication network. Similar elements in other service areas ofthe premises are coupled together for network communication with eachother through data communication media to form one or more otherportions of the physical data communication network at the premises 21.The communication interface 15 in each lighting device 11 in aparticular service area will be of a physical type and configured tooperate in a manner that is compatible with the physical media andelectrical protocol(s) implemented for the particular service areaand/or throughout the premises 23. Although the communication interfaces15 are shown communicating to/from the network cloud via lines, such aswired links or optical fibers; some or all of the interfaces 15 may usewireless communications media such as optical or radio frequencywireless communication. Also, although the examples in FIG. 1A show mostof the lighting devices 11 having one communication interface, some orall of the lighting devices 11 may have two or more communicationsinterfaces to enable data communications over different media with thenetwork(s) and/or with other devices in the vicinity.

The various portions of the network in the service areas in turn arecoupled together to form a data communication network at the premises,for example to form a premises-wide local area network (LAN) or thelike. The overall premises network, generally represented by the cloud23 in the drawing, encompasses the data links to/from individual devices11 and any networking interconnections within respective areas of thepremises where the devices 11 are installed as well as the LAN or otherpremises-wide interconnection and associated switching or routing. Inmany installations, there may be one overall data communication network21 at the premises. However, for larger premises and/or premises thatmay actually encompass somewhat separate physical locations, thepremises-wide network may actually be built of somewhat separate butinterconnected physical networks represented by the dotted line clouds.The LAN or other data network forming the backbone of system network 23at the premises 21 may be a data network installed for other datacommunications purposes of the occupants; or the LAN or otherimplementation of the network 23, may be a data network of a differenttype installed substantially for lighting system use and for use by onlythose other devices at the premises that are granted access by thelighting system elements (e.g. by the lighting devices 11).

Hence, there typically will be data communication links within a room orother service area as well as data communication links from the lightingdevices 11 in the various rooms or other service areas out to widernetwork(s) forming the data communication network 23 or the like at thepremises 21. Devices 11 within a service area can communicate with eachother, with devices 11 in different rooms or other areas, and in atleast some cases, with equipment such as 27 and 29 outside the premises21.

Various network links within a service area, amongst devices indifferent areas and/or to wider portions of the network 23 may utilizeany convenient data communication media, such as power lines wiring,separate wiring such as coax or Ethernet cable, optical fiber,free-space optical, or radio frequency wireless (e.g. Bluetooth orWiFi); and a particular premises 21 may have an overall data network 23that utilizes combinations of available networking technologies. Some orall of the network communication media may be used by or made availablefor communications of other gear, equipment or systems within thepremises 21. For example, if combinations of WiFi and wired or fiberEthernet are used for the lighting system communications, the WiFi andEthernet may also support communications for various computer and/oruser terminal devices that the occupant(s) may want to use in thepremises. The data communications media may be installed at the time aspart of installation of the lighting system 10 at the premises 21 or mayalready be present from an earlier data communication installation.Depending on the size of the network 23 and the number of devices andother equipment expected to use the network 23 over the service life ofthe network 23, the network 23 may also include one or more packetswitches, routers, gateways, etc.

In addition to a communication interface 15 for enabling a lightingdevice to communicate via the network 23, some of the devices 11 mayinclude an additional communication interface, shown as a wirelessinterface 15W in the lighting device 11B. The additional interfaceallows other elements or equipment to access the communicationcapabilities of the system 10, for example, as an alternative userinterface access or for access through the system 10 to the WAN 25.

A host computer or server like 27 can be any suitable network-connectedcomputer, tablet, mobile device or the like programmed to implementdesired network-side functionalities. Such a device may have anyappropriate data communication interface to link to the WAN 25.Alternatively or in addition, a host computer or server similar to 25may be operated at the premises 21 and utilize the same networking mediathat implements data network 23.

The user terminal equipment such as that shown at 29 may be implementedwith any suitable processing device that can communicate and offer asuitable user interface. The terminal 29, for example, is shown as adesktop computer with a wired link into the WAN 25. However, otherterminal types, such as laptop computers, notebook computers, netbookcomputers, and smartphones may serve as the user terminal computers.Also, although shown as communicating via a wired link from the WAN 25,such a user terminal device may also or alternatively use wireless oroptical media; and such a device may be operated at the premises 21 andutilize the same networking media that implements data network 23.

For various reasons, the communications capabilities provided at thepremises 21 may also support communications of the lighting systemelements with user terminal devices and/or computers (not shown) withinthe premises 21. The user terminal devices and/or computers within thepremises may use communications interfaces and communications protocolsof any type(s) compatible with the on-premises networking technology ofthe system 10. Such communication with a user terminal, for example, mayallow a person in one part of the premises 21 to communicate with alighting device 11 in another area of the premises 21, to obtain datatherefrom and/or to control lighting or other system operations in theother area.

The external elements, represented generally by the server/host computer27 and the user terminal device 29, which may communicate with theintelligent elements of the system 10 at the premises 21, may be used byvarious entities and/or for various purposes in relation to operation ofthe lighting system 10 and/or to provide information or other servicesto users within the premises 21, e.g. via the interactive user interfaceportal offered by the lighting devices 11.

Returning now to the lighting devices 11, in the example of the system10, at least one of the intelligent lighting devices 11 includes a userinput sensor configured to detect user activity related to user inputswithout requiring physical contact of the user; and at least one of theintelligent lighting devices 11 includes an output component configuredto provide information output to the user. The drawings shows severaldifferent examples of these input/output elements.

By contrast, some of the lighting devices 11 may not have user interfacerelated elements. In the example of system 10 in FIG. 1A, each of thelighting devices 11A includes a light source 13, a communicationinterface 15 linked to the network 23 and a processor 17 coupled tocontrol the light source 13 and to communicate via the interface 15 andthe link to network 23. Such devices 11A may include lighting relatedsensors (not shown), such as occupancy sensors or ambient light color orlevel sensors; but the intelligent lighting devices 11A do not includeany user interface components, for user input or for output to a user(other than control of the respective light source 13). The processorsof devices 11A are configured (e.g. programmed in our example) tocontrol lighting operations, for example, to control the light sources13 of such devices 11A in response to commands received via the network23 and the interfaces 15.

For purposes of discussion, the drawing (FIG. 1A) shows three examplesof lighting devices 11B, 11C and 11D that have one or more userinterface components. Although three examples are shown, it is envisagedthat still other types of interface components and/or arrangementsthereof in various intelligent lighting devices may be used in anyparticular implementation of a system like the system 10 of FIG. 1A; andthe later more detailed example of FIG. 1B shows a device thatincorporates a combination of several different user input and outputcomponents. Any one intelligent lighting device that include componentsto support the interactive user interface functionality of the system 10may include an input sensor type user interface component, an outputtype user interface component, or a combination of one or more inputsensor type user interface components with one or more output type userinterface components.

With reference to FIG. 1A, each of some number of intelligent lightingdevice 11B at the premises 21 includes one or more sensors 31 (two inthe illustrated example). The lighting devices 11B can be in one or morerooms or other service areas at the premises 21. In the intelligentlighting devices 11B, each of the sensors 31 is configured for detectionof intensity of received light and to support associated signalprocessing to determine direction of incident light. A particularexample of a sensor 31 that can be used as an input device fordetermining direction and intensity of incident light received by thesensor 31 is a quadrant hemispherical light detector or “QHD” (see e.g.U.S. Pat. Nos. 5,877,490 and 5,914,487). The sensors 31 may detect lightin some or all of the visible portion of the spectrum or in otherwavelength bands, such as infrared (IR) or ultraviolet (UV). By usingtwo or more such sensors 31 in the same or a different lighting device11B illuminating the same service area, it is possible to detectposition of an illuminated point or object in three-dimensional spacerelative to known positions of the sensors 31. By detecting position ofone or more points over time, it becomes possible to track motion withinthe area illuminated by the device(s) 11B and monitored for user inputby their sensors 31, for example, as a gestural user input. Although twosensors 31 are shown on one lighting device 11B; there may be moresensors 31 in a lighting device 11B, or there may be a single sensor 31in each device 11B amongst some number of the lighting devices 11Billuminating a particular service area of the premises 21.

In the example, at least one of the devices 11B also includes a lightingrelated sensor 33. Although shown in device 11B for purposes ofdiscussion and illustration, such a sensor may be provided in any of theother lighting devices 11, in addition or as an alternative todeployment of the sensor 33 in a lighting device 11B. Examples of suchlighting related sensor 33 include occupancy sensors, device output(level or color characteristic) sensors and ambient light (level orcolor characteristic) sensors. The sensor 33 may provide a conditioninput for general lighting control, e.g. to turn on-off devices 11and/or adjust light source outputs. However, the sensor inputinformation from sensor 33 also or alternatively may be used as anotherform of user input, for example, to refine detection and trackingoperations responsive to signals from the sensors 31.

In an example of a user input related function, the signals from thesensors 31 in lighting devices 11B illuminating a particular room withinpremises 21 are processed to detect gestures of one or morepersons/users within the room. The lighting output from sources 13 ofthe devices 11 illuminating the area may be controlled responsive to thedetection of one or more predetermined user input gestures. Although notshown, one or more of the lighting devices 11B may also include a useroutput component, for example to provide an audio or video output ofinformation to the person or persons in the room.

Such gesture input together with lighting control and other informationoutput implement a form of interactive user interface. This interfacerelated operation includes selectively controlling a lighting operationof at least some number of the lighting devices as a function of aprocessed user input. The interface related operation may also includeeither control a non-lighting-related function as a function of aprocessed user input, or an operation to obtain and provide informationas a response to a user input as an output via the output component.

In the example of system 10, each of the intelligent lighting devices11C and/or one or more of the lighting devices 11D in one or more roomsor other service areas of the premises 21 support audio input and audiooutput, for an audio based user interface functionality. These inputcomponents may be provided in different lighting devices 11 than thosedeploying the output elements. Also, audio user interface components maybe provided in different devices lighting devices 11 than thosedeploying the video user interface components. For convenience, theaudio input and output components and the video input and outputcomponents are shown together in each of the intelligent lightingdevices 11C, one or more of which may be deployed with other lightingdevices in some number of the services areas within premises 21.

Hence, in the example of FIG. 1A, each intelligent lighting device 11Cand/or one or more of the lighting devices 11D includes an audio userinput sensor such as a microphone 35. Any type of microphone configuredto detect audio user input activity, for example, for speech recognitionof verbal commands or the like, may be used; and some other types ofsensors may be used if they provide adequate response to audio input.Although the audio output may be provided in different devices 11; inthe example, each of the intelligent lighting devices 11C or 11D alsoincludes an audio output component such as one or more speakers 37configured to provide information output to the user. Where the speakeris provided in the same or a different device 11, there may be a singlespeaker 37 in each such device 11 or there may be some number ofspeakers in each respective lighting device 11.

The audio input together with lighting control and audio informationoutput implement a form of interactive user interface. Again, the userinterface related operation includes selectively controlling a lightingoperation of at least some number of the lighting devices 11 as afunction of a processed user input. The interface related operation mayalso include either control a non-lighting-related function as afunction of a processed user input, or an operation to obtain andprovide information as a response to a user input as an output via theoutput component.

Although shown for illustration purposes in the intelligent lightingdevice 11C, image-based input and/or output components may be providedtogether or individually in any others of the lighting devices 11 thatmay be appropriate for a particular installation. Although referred toat times as “video,” the image-based input and/or output may utilizestill image input or output or may use any appropriate form of motionvideo input or output. Hence, in the example of system 10, each ofseveral of the intelligent lighting devices 11D in one or more rooms ofthe premises 21 also supports image input and output for a visual userinterface functionality. Although related audio input and audio outputcould be implemented in other lighting devices, in the example, thedevices 11C also have the microphone 35 and the speaker 37 for the audiobased user interface functionality outlined above.

For the visual user interface functionality an intelligent lightingdevice 11C includes at least one camera 41. The camera 41 could be astill image pickup device controlled to capture some number of imagesper second, or the camera 41 could be video camera. By using a number ofcameras 41 to capture images of a given service area, it is possible toprocess the image data to detect and track user movement in the area,for example, to identify user input gestures. The multiple cameras 41could be in a single lighting devise 11D or could be providedindividually in two or more of the lighting devices that illuminate aparticular room or other service area. The image capture may alsosupport identification of particular individuals, e.g. via processing ofimages for face recognition, and associated customization of gesturerecognition and/or user responsive system operations.

The visual output component in the lighting device 11D is a projector43, such as a pico projector, in this example. The visual outputcomponent may take other forms, such as an integral display as part ofor in addition to the light source. Returning to the example of FIG. 1A,the projector 43 can present information in a visual format, forexample, as a projection on a table or a desk top or a wall or thefloor. Although shown in the same device 11D as the camera 41, theprojector 43 may be in a different intelligent lighting device 11. Also,the projector may be provided in a device 11 in an area that does notutilize a camera 41 for the user input sensor. For example, theprojector 43 may be in a device or in a service area with another device11 that utilizes a microphone (35) or the like as an audio sensor forspoken user input in an area that may also use sensors such as 31 in oneor more devices 11B to detect gestural inputs.

The combination of image-based input together with lighting control andimage-based and/or audio information output implement a form ofinteractive user interface. Again, the user interface related operationincludes selectively controlling a lighting operation of at least somenumber of the lighting devices 11 as a function of a processed userinput. The interface related operation may also include either control anon-lighting-related function as a function of a processed user input,or an operation to obtain and provide information as a response to auser input as an output via the output component.

In the example, one or more of the processors 17 in the intelligentlighting devices 11 are configured to process user inputs detected bythe user input sensor(s), such as the visual sensors 31, 33, 41,microphone(s) 35 and/or light sensors 33. Of course, other non-contactsensing technologies may be used (e.g. ultrasound) instead of or incombination with the input sensors discussed above. The processing ofsensed user inputs may relate to and control operations of the lightingdevices 11 in one or more areas of the premises 21. For example, theprocessing may detect spoken commands and/or relevant gestural inputsfrom a user to control lighting devices 11 in an area in which the usercurrently is located, e.g. to turn lights ON/OFF, to raise or lowerlighting intensity, to change a color characteristic of any tunablelighting devices 11 and/or various combinations of such changes. Asother examples, state changes may include changes of any one or anycombination of: distribution shape, spectral content (without changingcolor), aperture and/or fixture shape/size, fixture aim, color and/orluminance uniformity across fixture output, etc. Changes in lightoutput(s) in response to detected user inputs may also produce arepeating pattern or other sequence of changes in any one or more of theexamples or still other lighting related parameters, e.g. so as toconvey information or direct attention or to provide a desired variablelighting effect (such as a variable color ‘light show’ or moodlighting). Changes in the lighting in the occupied area of premises 21in response to such sensed user inputs would provide the user with avisual cue as part of the interactive user interface functionality. Theuser inputs also may be processed to control lighting devices 11 servingother areas of the premises 21. In addition to lighting controlfunctions, such as mentioned here by way of example, one or moreprocessors 17 in the intelligent lighting devices 11 may be configuredto process user inputs so as to enable the system 10 to obtain andpresent requested information to a user at the premises 21. By way of anexample of such additional operations, the system 10 may also enable useof the lighting devices 11 to form an interactive user interface portal,for access to other resources at the premises 21 (e.g. on userscomputers in other rooms at the premises) and/or access to outsidenetwork resources such as on server 27 or a remote terminal 29 (e.g. viathe WAN 25).

Although shown for illustration purposes in the intelligent lightingdevice 11D, any one or more of the lighting devices 11 may include asensor 39 for detecting operation of the lighting source 13 within therespective device 11. Such a sensor 39 may sense a temperature of thesource 13 or of other component of the device 11D, or a sensor 39 maysense an optical output of the source 13 (e.g. level or colorcharacteristic). The sensor 39 essentially provides feedback as to thestate of the source 13 or other component of the device 11D, which maybe used as part of the general control of the lighting device(s) 11. Byway of an example, where the performance of the source may have aneffect on sensing of user inputs, e.g. when a device 11B or 11C in aparticular service area optically detects gestures or other visual userinputs, source related feedback from sensor 39 may be used to adjustoutput of the source 13 in one or more of the devices illuminating thearea in a manner intended to assist in the detection of the visual userinput (e.g. to ensure adequate illumination for gesture detection).

In a system such as system 10 of FIG. 1, the lighting devices 11incorporate the elements and provide processing to support aninteractive user interface, for example, that need not require the userto touch or otherwise physically contact an element of the system. Theuser also need not have or operate a separate device, such as asmartphone or other portable terminal device. The lighting devicesthemselves implement the interactive user interface to the lightingsystem, and the user interacts with the lighting system via the lightingdevices 11.

The system 10, however, may also include or support communications forother elements or devices at the premises 21, some of which may evenoffer alternative user interface capabilities instead of or in additionto the interactive user interface supported by the lighting devices 11.The above-incorporated related applications, for example, disclose userinterface elements of the lighting system that are interconnected to thedata communication network of the system. Those applications alsodisclose standalone sensors of the lighting system that areinterconnected to the data communication network of the system, at leastsome which may perform sensing functions analogous to those of sensors33 and/or 39 in the system 10.

As disclosed in the above-identified U.S. application Ser. No.13/964,564, a system similar to the system 10 of FIG. 1 may also supportwireless communication to other types of equipment or devices at thepremises 21, to allow such other equipment or devices to use the network23 and/or to communicate with the lighting devices 11. By way ofexample, present drawing FIG. 1 therefore shows one of the lightingdevices including a wireless communication interface 15W, for such apurpose. Although shown in 11B, such an interface 15W may instead or inaddition be provided in any of the other lighting devices 11 in thesystem 10. Of note for purposes of the present discussion of userinterface techniques, the wireless link offered by the wirelesscommunication interface 15W allows the system 10 to communicate withother user interface elements at the premises 21 that are not includedwithin lighting devices 11 but which may be used in addition or as asupplement to the lighting device-centric user interface that isotherwise the focus of the present discussion. Although there may be anyof a wide range of such other types of user interface elements at anygiven premises 21, the drawing shows two examples, a remote control 47as an additional input device and a television or monitor 49 as anadditional output device. The wireless link(s) to devices like 47 and 49may be optical, sonic (e.g. speech), ultrasonic or radio frequency, byway of a few examples.

Any of the various system elements may be implemented using a PC likeapproach based on any known or available microprocessor architecture,such as a Reduced instruction set computing (RISC) using an ARMarchitecture, as commonly used today in mobile devices and otherportable electronic devices, or a microprocessor architecture morecommonly used in computers such as an instruction set architecture(ISA), like those used in Intel microprocessors and the like. Themicroprocessor based approaches are discussed by way of examples, withrespect to FIG. 1B; however, other processor implementations may beused, such as based on a Peripheral Interface Controller (PIC) or othermicrocontroller architecture. Alternative intelligent architectures forthe intelligence of the devices, however, will still include appropriatecommunication interfaces and couplings for light sources and may includeother standardized ports for connections of sensors, user input/outputdevices, etc.

Turning now to the example of FIG. 1B, the drawing depicts animplementation of an intelligent lighting device 11L using amicroprocessor centric architecture. At a high level, the fixture orother type of lighting device includes a light source, a power supplycircuit coupled to a power source, a processor, one or more memories anda communication interface; and the device will often include one or moresensors. To act as a portal, the lighting device will also have one ormore standard interface ports for attachment of elements for providingthe desired type of user interface. Each port may be for a hardwiredconnection to any compatible accessory or may provide a wireless link(e.g. WiFi, Zigbee or Bluetooth) for the accessory.

As an example of an implementation of the processors 17, discussed aboverelative to FIG. 1, the more detailed example of the lighting device 11Lincludes a microprocessor (μP) 123, which serves as the programmablecentral processing unit (CPU) of the lighting device 11L. The μP 223,for example, may be a type of device similar to microprocessors used inservers, in personal computers or in tablet computers, or insmartphones, or in other general purpose computerized devices. Althoughthe drawing shows a single μP 223, for convenience, the lighting device11L may use a multi-processor architecture. The μP 223 in the example isof a type configured to communicate data at relatively high speeds viaone or more standardized interface buses, represented generally by thebus/arrow 124.

The lighting device 11L includes one or more storage devices, which areaccessible by the μP 123 via the bus 124. Although the lighting device102 could include a hard disk drive or other type of disk drive typestorage device, in the example, the device 102 includes one or morememories 125. Typical examples of memories 125 include read only memory(ROM), random access memory (RAM), flash memory and the like. In thisexample, the memory or memories 225 store executable programming for theμP 123 as well as data for processing by or resulting from processing ofthe μP 123.

As in earlier examples, the intelligent lighting device 11L includes alight source 13. The source 13 may be such as an existing fixture orother luminaire coupled to the other device components, or the source 13may be an incorporated source, e.g. as might be used in a new design orinstallation. The source 13 may be any type of source that is suitableto the illumination application (e.g. task lighting, broad arealighting, object or personnel illumination, information luminance, etc.)desired for the space or area in which the particular device 11L is orwill be operated. Although the source 13 in the device 11L may be anysuitable type of light source, many such devices will utilize the mostmodern and efficient sources available, such as solid state lightsources, e.g. LED type light sources.

Power is supplied to the light source 13 by an appropriate driver 131.The source driver 131 may be a simple switch controlled by the processorof the device 11L, for example, if the source 13 is an incandescent bulbor the like that can be driven directly from the AC current. Power forthe lighting device 11L is provided by a power supply circuit 133 whichsupplies appropriate voltage(s)/current(s) to the source driver 131 topower the light source 111 as well as to the components of the device11L. In the example, the power supply circuit 133 receives electricityfrom alternating current (AC) mains 135, although the lighting devicemay be driven by a battery or other power source for a particularapplication. Although not shown, the device 11L may have or connect to aback-up battery or other back-up power source to supply power for someperiod of time in the event of an interruption of power from the ACmains 135.

The source driver circuit 131 receives a control signal as an input fromthe processor 123 of the device 11L, to at least turn the source 13ON/OFF. Depending on the particular type of source 13 and associateddriver 131, the processor input may control other characteristics of thesource operation, such as dimming of the light output, pulsing of thelight output to/from different intensity levels, color characteristicsof the light output, etc. If the source and/or driver circuit have thecapability, the driver circuit 131 may also provide some informationback as to the operation of the light source 13, e.g. to advise theprocessor 123 of the actual current operating state of the source 13.

The lighting device 11L also includes one or more communicationinterfaces 141. The communication interfaces at least include aninterface configured to provides two way data communication for the μP(and thus for the device 11L) via the network 23. In the example of FIG.1A, each communication interface 141 is of a type having a bus interfaceto enable the interface 141 to communicate internally with the μP 123via the bus 124. The interface 141 that provides the communication linkto the data communications network 23 enables the μP 123 to send andreceive digital data communications through the particular network 23.As outlined earlier, the network 23 may be wired (e.g. metallic oroptical fiber), wireless (e.g. radio frequency or free space optical),sonic or ultrasonic, or a combination of such network technologies; andthe interface 141 to that network 23 in a particular installation of thedevice 11L will correspond to the most advantageous network available(based on considerations such as cost and bandwidth) at the location ofthe installation. Some devices 11L may include multiple interfaces tothe network 11L; and or some devices 11L may include interfaces(analogous to the interface 15W discussed earlier) for communicationwith other equipment in the vicinity.

A device like 11A in the FIG. 1 example may have just the components ofdevice 11L discussed to this point in our more detailed example.However, for implementations of devices like 11B to 11C in the FIG. 1example may have one or more user input sensors configured to detectuser activity related to user inputs and/or one or more outputcomponents configured to provide information output to the user.Although the input and output elements and/or such elements of differenttypes, for convenience, the device 11L shown in FIG. 1B includes bothinput and output components as well as examples of several types of suchcomponents.

In the example, the intelligent lighting device 11L includes a number ofoptical sensors, including one of more of the sensors 31 configured fordetection of intensity of received light and to support associatedsignal processing to determine direction of incident light. Theintelligent lighting device 11L in this example also includes anothertype light sensor, such as a sensor 33 or 39. Although only one circuit143 is shown for convenience, the device 11L will include appropriateinput/output interfaces to operate and receive signals from theapplicable sensors 31, 33 and 39 included in the particularimplementation of the device 11L.

A sensor such as 31, 33 or 39 typically includes one or more physicalcondition detectors, which form the actual device that is responsive tothe particular condition to be sensed. The detector(s) may receive adrive signal; and in response to the sensed condition, the detector(s)produces one or more signals having a characteristic (e.g. voltagemagnitude, current or frequency) that is directly related to acharacteristic level of the sensed condition. A sensor such as 31, 33 or39 also includes a detector interface circuit that provides any drivesignal that may be needed by the particular device type of physicalcondition detector. The detector interface circuit also processes theoutput signal from the detector to produce a corresponding output, in astandardized format.

The sensor I/O circuit 143 in turn provides the input and outputinterface to couple the particular sensor(s) 31, 33 or 39 with the othercomponents of the intelligent lighting device 11L. On the side logicallyfacing the bus and processor, the sensor I/O circuitry 143 in theillustrated architecture provides a bus interface that enables the μP123 to communicate with the respective I/O interface circuit 143 via thebus 124. The a port for coupling the circuit 143 to the bus 124 may bein accordance with a standard, such as USB. Although not shown, thesensor I/O circuit 143 may fit a standard interface port on the boardforming the ‘brain’ and communication portion of the device 11L; and/orthe sensor I/O circuit 143 may provide physical and electricalconnections as well as a protocol for the interface with the applicablesensor such as 31, 33 or 39 in accordance with a standard, to allow useof sensors by different manufacturers.

The description of the sensors and I/O circuitry are given by way ofexample, and actual implementations may use somewhat differentarrangements. For example, the detector interface circuit referred toabove as part of the sensor may be incorporated in the applicable sensorI/O circuit 143. Each of the circuit(s) 143 may be configured to providethe electrical interface for one, two or more of the respective sensorsvia the associated coupling(s).

In the example, the intelligent lighting device 11L includes amicrophone 35, configured to detect audio user input activity, as wellas an audio output component such as one or more speakers 37 configuredto provide information output to the user. Although other interfaces maybe used, the example utilizes an bus connect audio interface circuitthat is or includes and audio coder/decoder (CODEC), as shown at 145.The CODEC 145 converts an audio responsive analog signal from themicrophone 35 to a digital format and supplies the digital audio to theμP 123 for processing and/or a memory 125 for storage, via the bus 124.The CODEC 145 also receives digitized audio via the bus 124 and convertsthe digitized audio to an analog signal which the CODEC 145 outputs todrive the speaker 37. Although not shown, one or more amplifiers may beincluded to amplify the analog signal from the microphone 35 or theanalog signal from the CODEC 145 that drives the speaker 37.

In the example, the intelligent lighting device 11L also includes acamera 41, configured to detect visible user input activity, as well asan image (still or video) output component such as a projector 43,configured to provide information output to the user in a visual format.The lighting device with also include appropriate input signalprocessing circuitry and video driver circuitry, for example, as show inthe form of a video input/output (I/O) circuit 147. The interface(s) toeither one or both of the camera 41 and the projector 43 could be analogor digital, depending on the particular type of camera and projector.The video I/O circuit may also provide conversion(s) between image dataformat(s) used on the bus 124 and by the μP 123 and the data or signalformats used by the camera 41 and the projector 43.

The actual user interface elements, e.g. speaker and/or microphone orcamera and/or projector, may be in the lighting device 11L or may beoutside the device 11L with some other link to the fixture. If outsidethe lighting device 11L, the link may be a hard media (wire or fiber) ora wireless media.

The device 11L as discussed above and shown in the drawing includes userinterface related components for audio and optical (including image)sensing of user input activities. That intelligent lighting device alsoincludes interface related components for audio and visual output to theuser. These capabilities of the device 11L and the system 10 support aninteractive user interface through the lighting device(s), for example,to control lighting operations, to control other non-lighting operationsat the premises and/or to provide a portal for information access (wherethe information obtained and provided to the user may come for otherequipment at the premises or from network communications withoff-premises systems).

For example, the device 11L and/or the system 10 can provide a voicerecognition/command type interface via the lighting device and networkto obtain information, to access other applications/functions, etc. Forexample, a user in the lighted space can ask for information such as astock quote or for a weather forecast for the current or a differentlocation. The user can ask for the system to check his/her calendarand/or the calendar of someone else and can ask the system to schedule ameeting.

In an initial implementation, the speech is detected and digitized inthe lighting device 11L and is processed to determine that the lightingdevice 11L has received a command or a speech inquiry. For an inquiry,the lighting device 11L sends a parsed representation of the speechthrough the lighting system 10 (and possibly an external network 25) toa server or the like with full speech recognition capability. The serveridentifies the words in the speech and initiates the appropriate action,for example, to obtain requested information from an appropriate sourcevia the Internet. The server sends the information back to the lightingdevice 11L (or possibly to another device) with the appropriate outputcapability, for presentation to the user as an audible or visual output.Any necessary conversion of the information to speech may be done eitherat the server or in the lighting device, depending on the processingcapacity of the lighting device. As the processing capacity of lightingdevices increases, some or all of the functions of the server in thisexample may be shifted into the lighting devices.

The lighting device 11L and the system 10 may provide similar servicesin response to gestural inputs, detected via sensors 31, one or morecameras 41 or a combination of sensors and cameras. Also, system thatinclude both audio and optical input components can respond tocombinations of speech and gestural inputs. Systems that include bothaudio and video output components can present information to the user(s)in various desirable combinations of audio and image or video outputs.

With an approach like that outlined above, the lighting system maysupport a broad range of applications or functions often performed viaother user terminal devices. For example, the user may be able to postto social media, access social media, send messages via mobile message(e.g. text) or instant messaging or email. The system with the interfaceportal enables the lighting system/service provider or some otheroperator of the system 10 to offer other services, such as informationaccess and personal communication. The lighting device may detect whenthe user enters the area and provide notices to appropriate ‘friends’ orthe like.

The discussion above has outlined the structure and configuration oflighting devices 11 and systems 10 of such devices as well as severaltechniques for implementing a lighting device-centric interactive userinterface that does not require the user physically touch a particularelement of the system or require the user to have and operate anadditional device. The interactive user interface relies instead onnon-contact sensing and appropriate output capabilities integrated intoone or more intelligent lighting devices. The user interface could beimplemented via processing by as few as one of the lighting devices 11.However, many installations will take advantage of processing by asubstantial number of the intelligent lighting devices 11. For complexoperations, such as processing of audio or optical inputs to detectspeech or gestural user inputs respectively, it may also be advantageousto perform some or all of the relevant processing using a distributedprocessing technique.

A variety of techniques for distributed processing that may beimplemented in the context of a lighting system like system 10 of FIG.1A. Distributed processing, for example, may enable use of availableprocessor and/or memory resources of a number of intelligent systemelements to process a particular job relating to user interfaceinteraction via one or more of the lighting devices 11. Anotherdistributed processing approach might entail programming to configuretwo or more of the intelligent system elements, e.g. lighting devices11, to implement multiple instances of a server functionality in supportof the interactive user interface capability, with respect to clientfunctionalities implemented on intelligent lighting devices. It may behelpful to consider some more specific examples of distributedprocessing operations. The process flow shown in FIG. 2 represents asimple example of a first type of procedure for distributed processing,which may be implemented in a lighting system like that of FIG. 1A.

In the example, a first lighting system element has a processing job toperform. The element may be any intelligent element of the system 10,although for purposes of a specific example to discuss, we will assumethat the element that has the processing job or task is one of theintelligent lighting devices 11 of FIG. 1A, and is therefore identifiedas device 1 in FIG. 2. At step S1, the device 1 recognizes that it maybe prudent to seek help to perform the task at hand, in this case, usingresources of others of the intelligent system elements.

The device 1 can perform at least some tasks utilizing the element's owninternal processor and memory. For example, a lighting device 11typically will be able to receive and appropriately process a lightingcommand, e.g. to set a light level and/or to set an associated colorcharacteristic of the device's light source, to adjust its operationallight output as commanded, without the need for resources of otherintelligent elements of the system. However, other tasks may morereadily lend themselves to resource sharing type distributed processing.Some such tasks with a potential for distributed processing may call formore processing or memory resources than readily available within thelighting device 1 (e.g. without compromising core lighting functions ofthe device). Tasks with a potential for distributed processing typicallywill be tasks that can be handled in some reasonable fashion by somenumber of individual elements, e.g. can be readily split into sub-tasksfor processing and/or storage in different elements, although there maybe some tasks that by the nature of the processing or storage involvedcannot readily be spilt amongst multiple elements. Some tasks mayrequire faster completion than the lighting device 1 alone can providewith only its own resources and therefore best implemented viadistributed processing. Conversely some resource intensive tasks may berelatively insensitive to time-to-completion and amenable to widerdistribution for processing (e.g. processing of audio, image or videodata).

In general, distributed processing tasks may relate to lighting systemoperations, general processing tasks associated with the system and/ortasks for other parties; although for purposes of this discussion, oneor more such tasks relate to the interactive user interface provided bythe lighting devices 11 of system 10. Some of these tasks may beimplemented by a lighting device 11 operating as a server, whereas othertasks may originate at a device 11 acting individually, e.g. upon itsreceipt of a particular sensed input. Examples of processing tasks ofthe system 10 that may be amendable to resource sharing type distributedprocessing include processing video inputs or processing other opticalinputs or audio inputs so as to detect user input activities, either fora lighting control operation in response to user input or for some othersystem function or feature (e.g. to access information or a non-lightingcontrol function in response to the user audio or video input). Similardistributed processing may apply to a task for an outside party, such asa task that might entail processing sensor, audio or video input datafrom the system elements for delivery to an outside party, either on aregular basis or in response to a specific request/instruction from theoutside party. Similar processing may be handled on a distributedprocessing basis within the system, to process data that a device 1receives from outside the system 10.

For purposes of further discussion, we will assume that the task for theresource sharing type distributed processing is a processing task thatin some way supports the interactive user interface provided by theintelligent lighting devices. Such processing, for example, may apply toprocessing of inputs or may involve processing of data or informationfor output to the user (e.g. decoding or decompressing data).

From the various factors involved in processing the particular task athand, in the processing flow of FIG. 2, the device 1 will recognize thatthe task is one that is appropriate for distributed processing, e.g.involving processor or memory intensive operations and/or not timecritical, etc. Also, based on characteristics of the job, e.g. source,lighting/non-lighting function, time sensitivity, or the like, thedevice 1 will assign a relative priority value or level to theparticular processing job. The programming and/or the protocols used forsignaling between system elements that may be involved in distributedprocessing in the system 10 can define an appropriate format and rangeof values for a job priority level parameter.

The lighting device 1 will be in communication with at least some numberof other intelligent elements of the lighting system 10, referred to inthis process flow example as neighbors of the device 1. The neighborelements may be other lighting devices 11, or the neighbor elements maybe other types of intelligent elements that are part of or communicatingvia the lighting system 10, such as additional interface devices (notshown) or standalone intelligent sensors (not shown).

At step S2, the lighting device 1 queries other intelligent systemelements, i.e. the neighbors in the example, essentially to request helpin performing the processing task or job. The queried neighbors mayinclude any number of other elements of the system 10. A small group ofneighbors, for example, might be those elements logically associatedwith the device in some small group or sub-network, such as elements inthe same room or other service area sub-network. The queried neighborsmay include all system elements on the system 10 or any sub-set ofelements between the smallest size group and the complete set. Asdiscussed more later, the sending device 1 may pick and choose which ofits ‘neighbors’ from any particular grouping to query with regard to thecurrent job or task, based on information about element performancelearned from earlier distributed processing of other tasks, knowncapabilities of the other elements and/or requirements for the task athand.

The exemplary distributed processing procedure includes learningfeatures, for the device that is distributing the job and for theneighbors that respond to queries or requests to contribute resourcesfor distributed job processing and/or that actually contribute theirresources to distributed job processing. The learning process on eachside of the distributed processing, job sub-task distribution as opposedto offering resources and performing an allocated sub-task, help thevarious system elements to adapt and optimize the distributed processingoperations over time. As will be discussed at various stages of ourdescription of the exemplary processing flow of FIG. 2, information thathas been learned from distributed processing of prior jobs informs thevarious elements in their decisions or responses at various stages ofthe process. Optimization may also involve some randomization.

For learning purposes, each intelligent system element configured todistribute portions of a task may establish, maintain and store alearning table for the distribution function; and each intelligentsystem element configured to offer resources to another intelligentsystem element and if instructed contribute such resources to adistributed processing operation may establish, maintain and store alearning table for such in-bound query response and sub-task processing.Of course, many of the intelligent system elements may play both rolesduring processing of different jobs over a period of time and may learnabout both sides of the distributed processing. An intelligent systemelement configured to participate on both sides of the distributedprocessing may maintain learned data about both types/sides of theoperations, either in two tables or in a combined table. If separatetables are used, each table may be adjusted in response to a change inthe other, under appropriate circumstances.

In general, learning entails analysis of performance by an elementand/or by other elements involved in handling of each distributedprocessing job determine to distributed processing metrics ofperformance. Examples of learned performance parameters that may beassessed in selecting other neighbor elements during the taskdistribution include turn-around time or turn-around time per unit ofprocessed data, number or percentage of dropped packets, average amountof memory resources offered (e.g. bytes of storage) and/or amount ofprocessing resources offered (e.g. in units related to data to beprocessed, number of processing cycles or average processing rate)and/or actually provided, during some number of prior distributed jobprocessing operations. Examples of learned performance parameters thatmay be assessed in determining how to respond to a new inquiry fordistributed processing assistance include amount of data processed, timerequired, resources used, delay incurred in processing of other tasks,or the like, for tasks distributed by the received device.

In general, the learned distributed processing metrics of performanceallows an element to prioritize one or more lists of neighbors/otherelements for use in making decisions and selections based on highestrelative ranking on the applicable list. For distribution, the device 1may select some number of the highest ranking neighbors. In contrast, anelement offering to take part in a distributed task may choose whetherto offer to help or how much if any of that element's resources to offerbased on the ranking of the particular requesting device 1, based onlearned distributed processing metrics of performance. With such anapproach, an element tends to select or respond most favorably to thehighest ranked element(s) in the particular prioritized listing, in aneffort to optimize operations.

When decisions in the process (e.g. FIG. 2) are made based on thelearned performance metrics about other elements, however, the elementmaking the decision can introduce a random variation in the decision,for example, to select or respond to a lighting device or other elementthat has not or seldom been chosen or favored at the particular decisionpoint in the past. As a result, the element making the selection orresponse will from time to time randomly select or favor another elementthat would otherwise appear as less than optimum based solely on thepreviously learned performance information. However, this allows theselecting or responding element to learn more about the randomly chosenelement for future processing purposes and update the parameters in thelearned table(s) for optimization of future distributed processingoperations. A random variation of this type, for example, may allow theelement making the decision to discover changes and adjust its learnedinformation accordingly, for better optimization of future distributedprocessing operations.

Returning to the process flow of FIG. 2, in a particularly intelligentimplementation of the distributed processing, the device with the taskto distribute can select among elements in some group or sub-group basedon performance data about elements in the group or sub-group learnedfrom prior job distribution operations for sending the query in step S2.The learned performance parameters for job distribution enables thedevice 1 to prioritize a list of neighbor elements for job distributionand to query some number of the highest priority elements likely tooffer and provide sufficient resources to handle the particular task athand. Only a few may be chosen from the high-end of the priority listfor a small task, whereas the sending device 1 may select more or all ofthe neighbors to query for a larger task. As the process is repeatedover time for multiple distributed processing tasks, the device 1 willtend to most often choose the other elements that are rated higher forperformance based on the learned performance parameters, for the querystep. Lower rated elements will be selected less often. However, thepriority for such selection for the query step S2 may change over timeas conditions at other elements change and the sending device 1 updatesits learned performance metrics accordingly; and the occasionalrandomization of the neighbor selection can enhance the process oflearning about changes.

The device 1 sends the query message through the network media used inthe relevant portion(s) of the data communication network 23, of thesystem 10, installed at the particular premises 21, to the neighborschosen initially for purposes of the inquiry about the current taskprocessing. The inquiry, for example, may be sent as a broadcast, sentas a multicast to selected neighbors or sent as individual data messagesto each of the selected neighbors, depending on the network media and/ordata communication protocols utilized by the particular systemimplementation.

The request message for the query in step S2 will include at least someinformation about the current job, including the assigned job prioritylevel. The information in the query, for example, may also providevarious metrics about the task at hand and/or the sub-tasks thereofbeing distributed to other elements. For example, such information mayindicate the type of processing involved, the type/format of the data tobe processed, any time constraints or deadlines for sub-task completion,the overall amount of data or the expected sub-divided amounts of datato be processed by recipient neighbors, or any other parameters aboutthe task that may be helpful in enabling the queried neighbors todetermine how to respond to the query. The information about the currentjob may also include a job or task identifier.

Each queried neighbor element will analyze the information about the jobfrom the query message it receives from the device 1 in comparison toits own resources, current data processing operations, status or thelike. For example, the receiving element may compare the priority of thetask that is to be distributed to the priority or priories of any of itsown tasks in progress or any distributed processing sub-tasks thereceiving element may already be working on for other source elements.The receiving element may also analyze factors about the task that is tobe distributed, versus what if any of its own resources that elementmight offer and allocate to the task, in view of its ongoing processingoperations and any expected higher priority tasks. For example, if thereceiving element is a lighting device 11, that receiving element may beable to offer some resources to handle part of the task but stillreserve sufficient resources to address a command to change a lightsetting if received while working on a part of the task.

Neighbor elements that do not have (or for various reasons will notoffer) resources may not respond to the query. Alternatively, suchunavailable neighbor elements may send responses, but their responses insuch cases would indicate that they are not offering resources to assistin performance of the distributed processing job currently offered bythe device 1. In the example, the device 1 will adjust its learned tableabout its neighbors to reflect any neighbors that do not offer to assistin the distributed processing job, e.g. to indicate other elements didnot respond or indicate any reason given in a response declining toparticipate.

Each receiving element that has resources available will set a requesttimeout and send a reply message back through the network to the device1 (S3). This period essentially is a time limit during which theneighbor will wait for further instructions about the job. However, ifthe timeout period expires (S4) without follow-up instructions about thejob from the device 1, then the neighbor will release the promisedresources at step S5, in this scenario, without having processed anypart of the task at hand. In this way, the unused resources areavailable for other uses by the neighbor element or for otherdistributed processing operations. After releasing the resources, theneighbor element will update its learning table about distributedprocessing offered by other elements, as shown at S6. In the timeoutscenario (that passed through S4), for example, the neighbor elementwill update its learned performance metric information about device 1 toreflect that device 1 did not send a sub-task to the neighbor elementafter the neighbor offered resources in response to the query. Theneighbor element can use such performance metric information in futureto adjust its responses to future queries from device 1.

Returning to step S3, as noted, at least the neighbors that have andwill offer available resources send back a reply message, which isreceived at the device 1. Each reply from a lighting device or otherelement offering to participate in the distributed processing operationwill include information about the resources of the neighbor elementwhich that element is offering to make available for sub-task processingof the currently offered job. Examples of such available resourceinformation include: processing power, memory, software/capability,reservation time, etc. Each reply may also indicate the relativepriority of any local task or prior distributed processing task that isalready in progress on the responding neighbor element. In this step S3,the requesting device 1 will receive similar replies from some number ofits neighbors, indicating whether or not the other intelligent systemelements have processing or memory resources available for theprocessing job. In our example, at least some of the replies fromneighbors offering available resources provide information about theresources that each other element offering to help in the distributedtask processing can make available. In the example, the device 1 willadjust its learned table about its neighbors to reflect those neighborsthat offered to assist in the distributed processing job and/or toreflect the resources each such neighbor offered in response to theinquiry sent in step S2.

In step S7, the device 1 with the task to distribute analyzes potentialcandidates for distributed processing of the task, for example, toprioritize a list of the neighbor elements that responded (respondents,in the drawing). The device 1 can prioritize the respondents based oninformation contained in the responses, for example, based oninformation about the resources each is offering and/or priority of anyother tasks the respondents are already processing. The device 1 canalso prioritize the respondents based on learned information regardingperformance metrics of the respondents that the device 1 selected andused to assist in prior distributed processing operations.

The device 1 in our example will also know the priority and requirementsof the data processing task that the device 1 is trying to distribute.From the prioritized list created in S7, the device 1 can now select anappropriate number of the respondents starting at the highest rank andworking down through the list to select a sufficient number of therespondents to provide the resources to meet the requirements of theparticular data processing task.

The device 1 essentially allocates portions of the processing job to theselected respondent elements. Hence, at step S8, the device 1 createswork packets for the selected respondents. By work packets here, we donot necessarily mean IP packets or the like; but instead we arereferring to sets of instructions and associated data for the portionsof the job that the device 1 allocates to the selected respondents. Forlarge processing jobs, for example, in a system using IP packetcommunications over the network media, each ‘work packet’ for a sub-taskallocated to a selected respondent may utilize some number of IP packetsaddressed to the particular respondent neighbor element. The device 1may send one, two or more work packets to each of the selectedrespondent neighbor elements. In our example, the distributing device 1stores a record of each work packet and an identifier of the neighborelement to which the device 1 assigned the particular work packet.

The work packets assigned and/or created for each selected respondentmay be tailored to the particular respondent. For example, respondentsoffering more processing or memory resources may be sent more of thedata to process. Respondent elements with particularly advantageouscapabilities (e.g. a video processor not currently engaged in anotherprocessing task) may receive task assignments particularly suited totheir capabilities. The allocations and associated work packet creationsalso may be adjusted based on the learning table. For example, if aparticular respondent has performed better in the past when handling asomewhat smaller data allocation, the device 1 may limit the dataallocation for that element accordingly.

In the process flow of FIG. 2, in step S8, the device 1 sends the workpackets to the selected respondents through the network communicationmedia of the lighting system 10. Although not shown for convenience, thesystem elements may be configured to require an acknowledgement of eachwork packet. In such an arrangement, a neighbor element would send anacknowledgement message back through the network to the distributingdevice 1. If no acknowledgement is received from a particular neighborelement, after some number of one or more re-tries, the distributingdevice 1 could select a lower priority neighbor from the list used instep S8 and try sending the undelivered work packet to the alternateneighbor in a similar fashion. Each work packet sent/delivered to aneighbor element will include a portion of the data to be processed forthe particular task as well as instructions as to how the data in thework packet is to be processed, essentially to enable each respondent toperform an allocated portion or sub-task of the distributed processingjob. Each work packet may include an identifier of the overallprocessing job and/or an identifier of the particular assigned sub-task.

At this point in the discussion, we will assume that each intelligentsystem element that receives a work packet for an allocated portion ofthe distributed processing job will successfully complete and returnresults for the portion of the job allocated thereto. Several scenariosin which work packets are dropped without sub-task completion will bediscussed later.

Hence, at this point in our process flow example, each of the neighborelements that the device 1 selected for a sub-task receives one or morework packets containing data and instructions for that sub-task as partof the communications in step S8. The element receiving the work packetperforms its allocated portion of the processing job on the receiveddata, in accordance with the instructions, using resources of theprocessor and/or memory of the receiving element of the lighting system(step S9). At step S10, each selected respondent neighbor element sendsa result of its sub-task processing back through the data communicationnetwork of the system 10 to the device 1. In our example, each of thework result packets sent back to the distributing device 1 includes anaddress or other identifier of the responding neighbor element thatperformed the sub-task as well as an identifier of the overall task/joband/or an identifier of the respective sub-task.

Upon sending sub-task results in step S10, each respondent neighborelement will release the resources utilized in processing the sub-task,at step S5. The resources become available again for other uses by theneighbor element or for other distributed processing operations. Afterreleasing the resources, the neighbor element again will update itslearning table about distributed processing (at S6), in this case, basedon the sub-task processing that the element performed for the device 1.In the completed sub-task scenario, for example, the neighbor willupdate its learned performance metric information based on analysis ofthe task of device 1 to reflect the size of the assigned sub-task, theamount of resources and/or time utilized, what if any other tasks of therespondent neighbor element were delayed during this distributedprocessing operation, or the like. The neighbor element can use suchlearned performance metric information in future to adjust its responsesto future queries from device 1.

Returning to the result transmission step S10, as a result of thetransmissions from the neighbors selected back in step S10, the device 1will receive processing results or the sub-tasks from other intelligentsystem elements. In step S11 in our example, the device 1 compiles thereceived results and checks the composite result to determine if anywork packets were dropped or if there are any readily apparent errors.Sub-task identifiers and/or a combination of the overall task identifierand the neighbor address/identifier may assist the device 1 in combiningsub-task results from the various participating neighbor elements intothe appropriate overall composite result. At this point in ourdiscussion, we will assume that no packets were dropped and no errorsare detected. Hence, the compiling of the results of the allocatedsub-task processing from the other system elements assisting in thecurrent distributed processing operation essentially determines anoverall result of the processing job. Processing by the device 1proceeds to step S12, in which the device 1 reports the overall result.The report function here is given by way of just one example of anaction that the device 1 may perform based on the overall result of theprocessing job. The report may be sent to a higher level processingelement or service, e.g. a higher level control service 57 or to anoutside system management device 27 or 29. As other examples, reportingthe result may involve taking some action in the device 1 and/or inother lighting devices or the like, accessing data via the network 25and presentation thereof to the user via the appropriate outputcomponent, sending a face or voice recognition result to an outsidedevice of another party, etc. Of course, the device or any other systemelement may act in any of a variety of other ways based on the overallresult of the distributed processing operation.

For purposes of discussion here, any number of the different types ofreporting or action functions will relate to support for functions ofthe interactive user interface that the system 10 provides via theintelligent lighting devices 11. At this point, it may be helpful toconsider a voice recognition task/job, by way of an example. The device1, in this case a lighting device such as 11C, 11D or 11L having amicrophone input obtains digitized audio data, which the system willprocess for speech recognition. The work packets sent at S8 includeportions of the digitized audio, e.g. sub-divided at apparent speechpauses between words. The neighbor elements process received digitalaudio data to recognize one or more words in the audio data segments.The returned work result packets represent words of recognized speech.Each returned result may include one or more words, depending on amountof the audio data sent to the respective neighbor element. The device 1compiles the received words into a string of words in an ordercorresponding to the original audio stream (prior to division thereoffor work packets).

At S12, the device 1 may simply send the string of words to anotherelement in or communicating with the system for further processing orthe like. Alternatively, the device 1 may itself perform additionalprocessing, e.g. to analyze the word string to recognize a command, inwhich case the device 1 can act in response to the command or forwardthe command to another element in or communicating with the system forfurther processing or the like.

For example, if the spoken command is a lighting command, the device 1acting as a controller can then instruct some number of lighting devicesto adjust light settings thereof, in the service area where the audiofor the spoken command was received as an input, based on the recognizedspeech command. As another example, if the recognized command is forobtaining access to other information, e.g. a request for a currentquotation of the price of a particular stock, the device 1 can format anappropriate query and send that query to a server for a stock service.In response, the device 1 receives data answering the inquiry, from thestock service server. If the lighting device itself has appropriateoutput capabilities, e.g. a speaker or a projector, the device 1 mayprovide the appropriate output to the user. Alternatively, the device 1may send the resulting answer information through the network 23 of thesystem to 10 a lighting device 11C, 11D or 11L with an audio or displayoutput capability for local presentation in the service area where theaudio for the spoken information request command was received as aninput.

Returning to the exemplary processing flow of FIG. 2, upon completion ofthe distributed processing job, e.g. upon reporting the overall resultin S12 in our example, the device 1 will also update its learning table(step S13) to reflect the performance of various other system elementswith respect to the just completed job. For example, the table may beupdated to reflect devices that did or did offer resources in responseto the query. The learning table may be updated to reflect successfulcompletion by some of the other/neighbor elements versus packets droppedor errors created by processing of sub-tasks by still others of theneighbor elements. As outlined earlier, the device 1 can utilize thelearning table updated in step S13 to improve its neighbor selections(e.g. at steps S1-S2 and steps S7-S8) in future distribution of jobsamongst its neighbors.

If sufficient resources are available and/or enough other elementsrespond, some or all of the work packets sent out at step S8 may beduplicated and sent to two or more of the selected respondent neighborelements, for redundant processing. When the device 1 compiles theresults at S11, it may receive duplicate sub-task processing results. Ifthe device 1 detects errors, in many cases, at least one of theduplicative sub-task processing results may be successful and free oferrors; and the device 1 can utilize the error free results and discardthe duplicate version that is subject to errors. In some cases, anelement that accepted a sub-task may not respond, at least in a timelyfashion. From the perspective of device 1, the work packet sent to suchan element has been ‘dropped.’ However, if another element assigned thesame sub-task successfully completes its processing of that sub-task,the device 1 can still compile the overall job result using thesuccessfully completed sub-task result from that other respondent.Hence, duplicative allocation of sub-tasks can improve likelihood ofsuccessful completion of the distributed processing task. However, insome cases, problems may still arise. In any of these cases, the updateof the learning table in step S13 will reflect such outcomes withrespect to the performance metric data stored in the table relative tothe respective neighbor elements.

Assume next that when the device 1 checks results in step S11, anddevice 1 determines that some portion of the job has not beensuccessfully completed. In this situation, the device 1 determines atstep S14 that some rework of the job is necessary. If capable, thedevice 1 may perform any additional processing needed itself. If not,however, then the device can again distribute some or all of thesub-tasks to other system elements. In our illustrated example,depending on the type and/or amount of further data processing requiredto complete the distributed processing task, processing flows from stepS14 back to S1 or S7 and from there through the other steps of theprocess, essentially as discussed above, to obtain distributedprocessing results to complete the overall data processing job.

There may be a variety of reasons why a sub-task is not successfullycompleted, and either the work packet is dropped or the results returnedto the device 1 are subject to errors. For example, some communicationmedia may be subject to communication-induced errors too extensive toaddress with routine error correction technologies. In other cases, somepart of the data network of the system 10 may be down or congested.However, in other cases, events at one or more of the selectedrespondent neighbor elements may result in a dropped work packet, asreflected in our exemplary process flow at steps S15 and S16.

Returning to step S9, the various neighbors that responded, wereselected and received work packets are processing data from the packetsin accordance with the associated data processing instructions. Theoverall processing job, and thus the sub-tasks thereof, will have anassigned priority. Other tasks handled by the various intelligent systemelements also have assigned priorities. At step S15, one of the systemelements that has been processing data from the work packets at S9 nowreceives (or internally generates) an interrupt in view of an apparentneed to perform some other task having a higher priority than theparticular distributed processing job. That element will suspend itsprocessing of the allocated sub-task and perform the processing for thehigher priority task. Depending on the resources and time taken for thehigher priority task, the element may be able to resume sub-taskprocessing after completing processing for the higher priority task andstill deliver its sub-task results within the timeframe set for theparticular distributed processing job. If not, however, then the systemelement will drop the processing of the work packet of the particulardistributed processing job (step S16), to process the higher prioritytask. In this later situation, the element will release the promisedresources at step S5. After releasing the resources, the neighborelement will update its learning table about distributed processingoffered by other elements, as shown at S6. In the interrupt scenario(that passed through S15), for example, the neighbor will update itslearned performance metric information about device 1 to reflect thatthe respondent neighbor element was unable to complete the sub-taskbefore dropping the packet to handle the interrupt for a higher prioritytask.

Although not discussed in detail, the device 1 may also process someportion data or otherwise perform some sub-task of the distributed jobbefore compiling the results. Alternatively, the device 1 itself may beinvolved as a respondent neighbor in another distributed processingoperation while it waits for responses from the respondent neighbors inthe job the device 1 distributed.

Although the discussion of FIG. 2 mainly focused on distributedprocessing amongst lighting devices 11, as noted earlier, the resourcesharing implemented by a process flow like the example of FIG. 2 maytake advantage of and share resources of any other type(s) ofintelligent elements that are part of or communicating via the lightingsystem 10. For example, if provided, the resource sharing typedistributed processing procedure may utilize resources of any additionaluser interface devices and/or standalone sensor devices. As anotherexample, the lighting system 10 may be able to use the memory and/or oneor more processors of a cooperative laptop, desktop computer,host/server computer or the like that is coupled to communicate via thedata communication media of the lighting system 10. The system also maysupport communications and/or interactions with a wide range of otherdevices within the premises 21 having some level of intelligence andhaving appropriate data communication capabilities, e.g. HVACcomponents, smart appliances, etc. If any of such other devices haveresources to share and are appropriately programmed, the lighting system10 may be able to use the memories and/or processors of such othercooperative devices. Conversely, some other types of lighting systemelements, computers coupled to the communication media and/or some othertypes of cooperative devices may be able (and permitted if appropriate)to request and obtain access to resources of the lighting devices,associated user interface devices and sensor devices available forsharing for distributed processing in a manner like that shown by way ofexample in FIG. 2.

In a system like that shown in FIG. 1, some functions may be implementedin the form of servers, for example, for central control, CO functionsin support of system commissioning and/or for central connectivity tooutside networks and equipment. Of note, some aspects of the userinterface provided by the system 10 through the intelligent lightingdevices 11 may involve serve functions and related client-servercommunications. Some or all of the intelligent lighting devices (andpossibly some other intelligent system elements) may have sufficientcapabilities to be able to run appropriate server programming inaddition to performing other processing tasks for the normal functionsof such elements. For example, one or more lighting devices 11A may beconfigured to operate as a server for overall control of a particularsystem function and/or for information access purposes, in addition tooperating as a lighting device. To the extent that a server task forsuch a centralized service is amenable to resource sharing typedistributed processing, the lighting device 11A also configured as aserver may distribute the server processing task to other elements, inessentially the manner described above relative to FIG. 2. Lightingdevices 11A are identified here for performing the server functions,since in the example of FIG. 1A, those devices 11A did not include anyof the user input/output components and thus did not directly processinguser inputs or information for output to the user. Any or all of theother types of lighting devices 11B, 11C, 11D or 11L, however, may runthe appropriate server programming.

Hence, the system 10 may implement another form of distributedprocessing with respect to server functionality, instead of or inaddition to the resource sharing approach outlined above relative toFIG. 2. Some or all of these server functions may have interactive userinterface implications, e.g. in deciding how to change light settings inresponse to gestural or speech inputs, obtaining and processingrequested information for presentation to a user, etc.

A single instance of a server running on one system element may at timesbe stressed by high processing demands. Also, a system that utilizes asingle server instance for a crucial system function or service may bevulnerable to interruptions, e.g. if there is a failure of the elementor of communication with the element running the server instance. Toaddress such possible concerns, a system 10 can run some number ofseparate instances of a particular server functionality, in parallelwith one another on multiple intelligent system elements. Also,different server functions may run on different system elements.

For a particular function, each server instance would utilize a copy ofthe relevant server programming and a copy of any data or databaseneeded for the particular system service. Use of multiple instances ofthe servers may also speed up response time when interacting withclients implemented on the other system elements.

To the extent that data used by the server functionality may change overtime of operation of the system 10, the sever instances would coordinatewith each other to update the copy of the data/database at or used byeach instance of the server, e.g. to maintain synchronism as betweenmultiple instances of the relevant data. FIG. 3 is a simplifiedillustration of such an arrangement. Alternatively, the data used by theserver functionality may be stored in a distributed manner acrossmultiple elements (e.g. as distributed hash tables) to minimize thesynchronization operations.

For discussion purposes, the drawing shows two of the lighting devices11A, there identified in FIG. 3 as devices 11A′ and 11A″. The devicesmay include elements 13 to 19, as in devices 11 of FIG. 1; therefore thesame reference numbers are used in FIG. 3 and detailed discussion ofthose elements is omitted here for convenience. The two intelligentlighting devices 11A′ and 11A″ implement a user interface related serverfunctionality 57. The server 57 in turn communicates through the network23 with other elements of the system acting as client devices; and thedrawing depicts the other/client element generically as the elements 61in this drawing. At least for most of the interactive user interfacefunctions under consideration here, most if not all of the other systemelements 61 will be other lighting devices, although there may be someother types of intelligent system elements acting as clients,particularly for some functions that do not directly impact theinteractive user interface offered through the lighting devices.

Hence, in the example, two of the lighting devices 11A′ and 11A″ runinstances 57A and 57B of server programming for execution by processors17 thereof. The server instances 57A, 57B configure those lightingdevices 11A′ and 11A″ to operate in a distributed processing fashion toimplement a server function 57 with respect to an overall user interfacefunctionality of the system 10 and to implement related servercommunications via the data communication network, generally representedagain by the cloud 23. The server program instances 57A, 57B arerepresented generally by icons similar to hardware devices such asserver computers; but the program instances 57A, 57B are actually serverprogramming stored in memories 19 for execution by the processors 17(hence, the servers 57A, 57B are shown in dotted line form). Althoughonly two instances of the server program are shown, there may be anyappropriate number of such instances for implementation of a particularfunction or service in a system of a particular size and/or complexity.

At least with respect to the particular overall processing function ofthe system 10 supported by the server program instances 57A, 57B, theserver program instances 57A, 57B interact with some number of otherintelligent system elements represented generically by the boxes at 61.The other elements can be any of the types of intelligent systemelements discussed above and will include at least a processor 63, amemory 65 and a communication interface, which may be similar tocomponents of the intelligent system elements discussed with regard tothe earlier drawings.

As shown in FIG. 3, various other intelligent system elements 61 willinclude client programming stored in memories 67 thereof for executionby the processors 63 of the other intelligent system elements 61, toconfigure each of the other intelligent system elements 61 to implementa client function with respect to the overall processing functionalityof the system supported by the server instances 57A, 57B. The clientprogramming 69 will also support related client communications with theserver function implemented by the instances of the server programming57A, 57B on the lighting devices 11A′ and 11A″ in our example. Hence,the drawing shows arrows through the network for client-servercommunications between the server instances 57A, 57B and the clients 69at the intelligent system elements 61.

In a multi-instance server implementation such as shown in FIG. 3, anyone server may be able to perform on its own to handle client-serverinteractions with one or more elements 61 independently of the otherserver instance(s), while each the other server instance(s)independently handles other client-server interactions. To the extentthat they relate to the same overall system function, however, they willoften use or process some of the same data. For example, if a particularoverall processing functionality of the system involves a database, allof the relevant server instances will manipulate that same database. Inour two instance server example, to insure that both instances of theserver programming 57A, 57B have access to the same state of thedatabase if or when necessary, the server instances 57A, 57B willcommunicate with each other through the data communication network 23 tosynchronize any separate copies of the database maintained by or for theindividual server instances 57A, 57B, as represented by the Sync arrowbetween the server instances 57A, 57B. Any appropriate datasynchronizing technique may be used.

The use of multiple server instances allows for server load distributionacross multiple hardware platforms of intelligent elements of thesystem. The use of multiple server instances may also provide redundancyin the event of impairment or failure of a system element orcommunications to an element executing one of the server instances.Various load distribution and/or fail-over techniques may be used.

FIG. 4 illustrates an example of an interactive user interfaceprocedure, where the interactive user interface is provided via theintelligent lighting devices, which may be implemented in the system 10and may involve one or more of the distributed processing techniquesdiscussed above. In this example, a user takes an action (at S20) thatrepresents an input. The action, for example, may involve speech, agesture or a combination of speech and gestural inputs. As outlined inthe discussions of FIGS. 1A and 1B above, one or more of the intelligentlighting devices 11 includes a user input sensor; and at step S21, sucha sensor detects the user activity as an input to the lighting device(S22) that includes the particular sensor.

In step S23 the processor(s) of one or more of the intelligent lightingdevices 11 process the signal or data from the sensor(s) representingthe detected user input to determine the appropriate response to bepresented via the interactive user interface. In some cases, theprocessing may be done by the processor of the lighting device thatdetected the input and/or by a lighting device acting as a server. Inother cases, the processing may involve processors of any number oflighting devices, e.g. using a resource sharing type distributedprocessing technique or a combination of processor operations in theclient lighting device and a server lighting device.

Upon completion of the processing, a determination is made as to how torespond and the decision is implemented by one or more of the lightingdevices at S24. This may be the same lighting device that detected theinput in steps S21 and S22 or the output-side operation at S24 may be atone or more other lighting devices instead or in addition to thelighting device that detected the input in steps S21 and S22.

Steps S25 to S27 represent different types of outputs to the user, someof which may be implemented in the lighting device that detected theuser input and others of which may be implemented in devices other thanthe lighting device that detected the user input. As discussed above,the user interface implemented by the lighting devices 11 can controllighting and control information output to a user via other outputcomponent(s). As shown at S25, each lighting device or devices(discussed re S24) intended to respond to the detected user inputadjusts its light output based on the processing of the detected userinput action. As shown at S26, one or more lighting devices having anappropriate information output component (e.g. speaker for audio outputand/or projector or display for image output) operates that informationoutput component to present information to the user. For example, theinformation may inform the user of the response (e.g. change in lightsetting) or any error or problem encountered attempting to respond, orthe information may be information obtained from some other source asdirected or requested by the user via the input activity. If the system10 includes other controlled components, the appropriate lightingdevice(s) activate one or more such components as the response to theuser.

FIG. 5 shows aspects of two procedures, each similar to that of FIG. 4,but in a signal flow format. In a first example shown by that drawing, auser takes an action (at S31) that represents an input. As noted above,the action may involve speech, a gesture or a combination of speech andgestural inputs. As outlined in the discussions of FIGS. 1A and 1Babove, one or more of the intelligent lighting devices 11 includes auser input sensor that detects the user activity as an input to thelighting device that includes the particular sensor. The signal or datafrom the sensor(s) that detect the user input is provided to theprocessor(s) of one or more of the intelligent lighting devices in stepS32.

The processor(s) of the one or more of the intelligent lighting devicesprocess the signal or data representing the detected user input todetermine the appropriate response to be presented via the interactiveuser interface (S33). At this point in the example of FIG. 5, we willassume that the input action from the user at S31 related to a desiredlighting change; therefore the processor(s) determine the appropriatechange as a result of the analysis in step S33. Hence, at step S34, theprocessor(s) provide instructions to one or more lighting devices toimplement the appropriate change. Although the lighting change may applyto only the lighting device that detected the user input or only in oneother device in the system; in our example, the processor(s) instructthe lighting device that detected the user input and some number ofother local lighting devices to make the change, e.g. to appropriatelychange an aspect of the overall illumination of the room or otherservice area within which the user made the input in a relatively steadystate manner or in a specified manner that changes over time. At stepS35, those lighting devices change their light outputs as instructed,which provides a lighting change output as the system response to theuser input to complete this first FIG. 5 example of user interfaceinteraction with the lighting system.

FIG. 5 also shows a second example of a user interface operationinvolving an information output responsive to the user input. In thissecond example, the same or a different user takes an action (at S41)that represents an input. As will be seen in later steps, a response tothis second user input via the interactive interface will lead to aninformation output (which may be instead of or in addition to a lightingchange). Again, for discussion purposes, the user activity may involvespeech, a gesture or a combination of speech and gestural inputs. Asoutlined in the discussions of FIGS. 1A and 1B above, one or more of theintelligent lighting devices 11 includes a user input sensor thatdetects the user activity as an input to the lighting device thatincludes the particular sensor. The signal or data from the sensor(s)that detect the user input is provided to the processor(s) of one ormore of the intelligent lighting devices in step S42.

The processor(s) of the one or more of the intelligent lighting devicesprocess the signal or data representing the detected user input todetermine the appropriate response to be presented via the interactiveuser interface (S43). At this point in the second example of FIG. 5, theprocessor(s) determine that a response will involve information to beobtained from other equipment. The information may be virtually any typeof information that the user may desire or that the system may otherwiseoffer to as a response to the user. Although the information may comefrom one or more of the lighting devices, it is also envisioned that theinformation may be obtained from other sources.

In the illustrated example, the information may come from a deviceimplementing a ‘local’ data storage, such as a computer of theenterprise or other occupant at the premises 21 or equipment at anotherassociated premises in communication with or via network 23. An exampleof such a data source might be an exchange server of the enterprise thatoperates its business or the like at the premises 23. Alternatively, theinformation might be available on any web site server or the likeaccessible via the public Internet.

If the analysis in step S43 determines that the information is on thelocal data storage device, then the processor(s) send a request for thedesired data to that storage device in S44A. The device in turn sendsback responsive data (e.g. requested information or an error message ifnot available) from the local storage in step S45B. Alternatively, ifthe analysis in step S43 determines that the information is availablethrough the Internet, then the processor(s) send a request for thedesired data, via the Internet, to the appropriate server or the likethat has the data. The processor(s) in turn receive responsive data(e.g. requested information or an error message if not available) viathe Internet, in step S45B. Although shown as alternatives, the localand Internet requests/responses may be used in a combined procedure,e.g. to request data from the Internet source in the event that aresponse from the local storage indicates that some or all of therequested information data is not locally available.

In these examples, after step 45A or 45B, the processor(s) have data forproviding an information output responsive to the user's latest inputactivity. Hence, at step S46, the processor(s) supply data inappropriate format(s) and any related instructions that may be requiredto enable particular lighting device(s) to control components thereofand thereby output the responsive information to the user. Depending onthe deployment of the output components in various lighting devices, theinformation may go back to only the lighting device that detected theuser input or only to another lighting device of the system in the sameor a different service area than the lighting device that detected theuser input. With some types of deployments, the same data or relatedportions of the data may go to the same device that detected the inputand other local devices. The drawing generally shows the various optionsfor distribution of the received information as arrows from step S46 toboth the lighting device that detected the user input and to other locallighting devices. By way of an example involving multiple devices as artof a response to one particular input, a device that detected an audioinput may also provide an audio output of some portion of theinformation, while another device in the vicinity outputs some portionof the information in a visible format. Light output by one or more ofthe lighting devices also may be controlled in some manner related tothe information output. Hence, at step S47, each lighting device thatreceives some or all of the responsive information controls one or morecomponents thereof to output the information as instructed, whichprovides the information output as the system response to the user inputto complete this second FIG. 5 example of user interface interactionwith the lighting system.

FIG. 6 is a flow chart of an interactive user interface procedure,specifically one involving speech input and recognition. Forconvenience, the lighting devices are referred to in the illustrationsof the steps in this drawing using the acronym LD.

In this example, a user speaks, as an input activity shown at step S51.As outlined in the discussions of FIGS. 1A and 1B above, one or more ofthe intelligent lighting devices 11 includes a microphone or other audioinput sensor that detects the user's spoken input activity as an audioinput to the lighting device that includes the particular sensor. Thesignal or data from the audio sensor(s) is provided to the processor ofthe lighting device (LD) that includes the sensor(s), and that processorperforms at least some initial processing in response to the detectedaudio in step S52.

All of the speech recognition could be implemented via distributedprocessing, in which case, the processor of the device would digitizeand possibly compress the audio and send the resulting digitizedrepresentation of the audio input to one or more processors forimplementation of distributed processing, to recognize the speech and todetermine further system action. In this example, however, theparticular lighting device (LD) has some basic speech recognitioncapability. Hence, as a result of the processing in S52, the processorof the particular lighting device (LD) determines in step S53 whether ornot the audio input represents a simple spoken command that the device(LD) can handle. If the audio input represents a simple command, thenthe processor of the particular lighting device (LD) follows the ‘Yes’branch from S53, and processing flows to step S54.

At S54, the processor of the lighting device (LD) determines whether ornot the command dictates a change of state of the output from the lightsource within that particular lighting device (LD). If so, the processbranches from step S54 to step S55, in which the processor makes theappropriate state change within the lighting device so that the lightsource of the device provides the appropriate light output responsive tothe user's spoken input command. If the determination at S54 is thatchange of light state within the particular lighting device (LD) is notappropriate for the system response to the user's spoken simple inputcommand (e.g. because the lighting device itself is already in theappropriate state or the command was directed to one or more otherlighting devices), then the process flow skips step S55 and branches tostep S56.

To this point in the example of FIG. 6, the processing functions and thedecisions have all been made by the processor of the one lighting device(LD) that received the audio input. At step S56, the processor of thatlighting device (LD) determines whether or not the command dictates achange of state of the output from the light source within any otherlighting device (LD) of the system 10. If the receiving device includesa table or the like indicating lighting states of the other devices,then the determination in S56 may be also be internal to the lightingdevice that received the audio input. In many implementations, however,the determination may involve some communication with and/or processingby one or more other elements of the system 10. For example, thereceiving device may query other devices and determine necessary statechanges if any by processing state information received from the otherdevices. Alternatively, the receiving device may work with anappropriate server implemented on the system (e.g. as in FIG. 3) toidentify other devices and the appropriate change if any that should bemade in the light output states of other lighting devices.

In one of these ways, a determination is made at step S56 as to whetheror not lighting state change is needed at any other lighting device ofthe system. If not, processing skips from S56 to S57 in which thelighting device that received the audio user input awaits a new userinput. However, if at S56, the device determines that the audio userinput dictates a lighting state change at any of the lighting device LD,then the receiving device sends an appropriate state change instructionto the other lighting device (step S58). Although not shown, each otherlighting device (LD) so instructed will make the appropriate statechange within the lighting device so that the light source of therespective other device provides the appropriate light output responsiveto the user's input. The state changes by the detecting lighting devicesand other lighting devices receiving such commands, for example, maycause some number lighting devices of the system to change state (e.g.ON/OFF, level, color characteristic(s), distribution, etc. as in theearlier discussion of changeable parameters), or cause lighting devicesto implement a pulsing or other sequential variation of one or more ofthe controllable lighting parameters among some number of the lightingdevices, etc. After sending any commands in S58, processing againadvances to step S57 in which the lighting device that received theaudio user input awaits a new user input.

Returning to consideration of step S53, assume next that the processorin the lighting device (LD) that received the user's spoken inputdetermines that the input does not represent a simple command that thereceiving device can handle itself. Hence, the processing branches fromS53 to step S59, in which the processor causes that device to refer thematter to other devices/equipment for processing. The receiving devicecould forward the digitized audio to one or more other devices; however,in the example, the receiving device parses the audio down to a word orcharacter string corresponding to the audio input. Hence, in step S59,the receiving device sends the string of parsed data to one or moreother devices on the data communication network 23 of the lightingsystem 10, that is to say, for further processing thereof in “thecloud.” Processing in the cloud will be performed on a distributedprocessing basis in a manner analogous to one or more of the proceduresoutlined above relative to FIGS. 2 and 3.

In step S60, one of the devices implementing the distributed processingof the parsed data in the cloud determines whether the audio inputindicates that the user was apparently speaking intentionally to thelighting system 10. If not, the processing branches from S60 to stepS61. In some cases, the system may take no action if the input was notdirected to the system. In the example, however, the distributedprocessing by the system determines characteristics of detected speech,such as intonation and/or mode; and the system may adjust lightingparameters (S62) accordingly, for example, ON/OFF state, level, colorcharacteristic(s), pulsing or other sequential variation among somenumber of lighting devices, etc. Lighting adjustment, for example, mayentail sending appropriate light state change instructions to applicablelighting devices, similar to the operations discussed above relative tostep S58; although at this point in the example, the adjustment mayentail sending a similar command back to the lighting device thatinitially received the audio user input. After the adjustment at S62,processing again advances to step S57 in which the lighting device thatreceived the user input awaits a new user input.

Returning to the processing at step S60, we will next assume that one ofthe devices implementing the distributed processing of the parsed datain the cloud instead determines that the audio input indicates that theuser was apparently speaking intentionally to the lighting system 10.Hence, at this point in our example, the process flow branches from stepS60 to step S63. To this point in the process, the deliberate input isnot a simple command to change lighting. In step S63, one of the devicesimplementing the distributed processing of the parsed data in the cloudtherefore determines whether or not the system 10 has data to provide aninformation response. If so, processing at step S64 changes a systemstate so as to process data representing the appropriate information, toenable a response output to the user as requested. Since the branch wasfrom S63 to S64, at this point, the system has any data for the desiredinformation output obtained from its own resources, e.g. informationbased on current system status, an indication of an error in processingthe audio input, etc.

Alternatively, if the processing at S63 results in determination thatthe system does not have the data to respond, then processing insteadbranches from S63 to S65. In step S65, one of the devices implementingthe distributed processing of the parsed data in the cloud instead sendsan inquiry to an appropriate storage, server or the like providing aslocal storage on the network 23 or on the Internet (analogous toobtaining the data from the local storage/exchange server or via theInternet in the example of FIG. 5). After obtaining the data fromanother source in step S65, processing at step S64 changes a systemstate so as to process data representing the appropriate information toenable a response output to the user as requested.

In either of the examples regarding steps S63 to S65, after processingat step S64, the cloud processing leads to a step S66 in which data foroutput is sent to the appropriate lighting device(s). One or morelighting devices outputs the data as information responsive to the userinput, again in a manner like that discussed above relative to FIG. 5.After sending the data to the device(s) that will output the informationto the user in step S66, processing again advances to step S57 in whichthe lighting device that received the user input awaits a new userinput.

The example of a process flow of FIG. 6 related to processing of anaudio input representing recognizable speech. A similar process may beused to handle gestural input detected by one or more cameras or othertypes of optical sensors. For gestural detection, however, more of theinitial image processing may be transferred to the cloud rather thanleading to a receiving device recognition of a simple command. Theseprocedures may also be adapted for handling of user inputs combiningrecognition of speech and gestures.

The examples outlined above may be modified, adapted or expanded in avariety of ways yet implement the interactive user interface via thelighting devices as descried herein. For example, instead of discreteinput and output components as in the examples of devices 11A to 11L inFIGS. 1A and 1B, some or all of the relevant components may be combined.For example, for a light device configured to implement a sky simulationwhen installed in a ceiling, the light source and an image output devicemay be combined in a single unit that operates in a manner analogous toa video display such as a computer monitor. As another example, audioinput and output components may use a single audio transducer, insteadof discrete microphone and speaker. Also, the speaker and/or themicrophone may be incorporated into a projector or other display unitthat provides an image type information output.

As shown by the above discussion, at least some functions of devicesassociated or in communication with the networked lighting system 10 ofFIG. 1A, such as elements shown at 27 and 29 (and/or similar equipmentnot shown but located at the premises 21), may be implemented withgeneral purpose computers or other general purpose user terminaldevices, although special purpose devices may be used. FIGS. 7-9 providefunctional block diagram illustrations of exemplary general purposehardware platforms.

FIG. 7 illustrates a network or host computer platform, as may typicallybe used to implement a host or server, such the computer 27. FIG. 8depicts a computer with user interface elements, as may be used toimplement a personal computer or other type of work station or terminaldevice, such as the terminal 29 in FIG. 1A, although the computer ofFIG. 8 may also act as a server if appropriately programmed. The blockdiagram of a hardware platform of FIG. 9 represents an example of amobile device, such as a tablet computer, smartphone or the like with anetwork interface to a wireless link, which may alternatively serve as auser terminal device like 29. It is believed that those skilled in theart are familiar with the structure, programming and general operationof such computer equipment and as a result the drawings should beself-explanatory.

A server (see e.g. FIG. 7), for example, includes a data communicationinterface for packet data communication via the particular type ofavailable network. The server also includes a central processing unit(CPU), in the form of one or more processors, for executing programinstructions. The server platform typically includes an internalcommunication bus, program storage and data storage for various datafiles to be processed and/or communicated by the server, although theserver often receives programming and data via network communications.The hardware elements, operating systems and programming languages ofsuch servers are conventional in nature, and it is presumed that thoseskilled in the art are adequately familiar therewith. Of course, theserver functions may be implemented in a distributed fashion on a numberof similar platforms, to distribute the processing load.

Also, a computer configured as a server with respect to one layer orfunction may be configured as a client of a server in a different layerand/or for a different function. In a similar fashion, a centralfunction or service 57A, 57B implemented as a server functionality onone or more lighting devices 11 with respect to clientprogramming/functionality of other intelligent system elements atpremises 21 may itself appear as a client with respect to a server in adifferent layer and/or for a different function such as with respect toa server 27.

A computer type user terminal device, such as a desktop or laptop typepersonal computer (PC), similarly includes a data communicationinterface CPU, main memory (such as a random access memory (RAM)) andone or more disc drives or other mass storage devices for storing userdata and the various executable programs (see FIG. 8). A mobile device(see FIG. 9) type user terminal may include similar elements, but willtypically use smaller components that also require less power, tofacilitate implementation in a portable form factor. The example of FIG.9 includes a wireless wide area network (WWAN) transceiver (XCVR) suchas a 3G or 4G cellular network transceiver as well as a short rangewireless transceiver such as a Bluetooth and/or WiFi transceiver forwireless local area network (WLAN) communication. The computer hardwareplatform of FIG. 7 and the terminal computer platform of FIG. 8 areshown by way of example as using a RAM type main memory and a hard diskdrive for mass storage of data and programming, whereas the mobiledevice of FIG. 9 includes a flash memory and may include other miniaturememory devices. It may be noted, however, that more modern computerarchitectures, particularly for portable usage, are equipped withsemiconductor memory only.

The various types of user terminal devices will also include varioususer input and output elements. A computer, for example, may include akeyboard and a cursor control/selection device such as a mouse,trackball, joystick or touchpad; and a display for visual outputs (seeFIG. 8). The mobile device example in FIG. 9 touchscreen type display,where the display is controlled by a display driver, and user touchingof the screen is detected by a touch sense controller (Ctrlr). Thehardware elements, operating systems and programming languages of suchcomputer and/or mobile user terminal devices also are conventional innature, and it is presumed that those skilled in the art are adequatelyfamiliar therewith.

Although FIGS. 7-9 in their present form show computers and userterminal devices, generally similar configurations also may be usedwithin other elements of the lighting system 10. For example, oneimplementation of the brain, communication and interface elements of alighting device may utilize an architecture similar to that of one ofthe computers or mobile terminals. As a more specific example, thepersonal computer type hardware in FIG. 8 (except for the keyboard,mouse and display) could serve as the brain and communication elementsof a lighting device, where the input/output interface I/O wouldinterface to an appropriate light driver and to any sensor(s) or otherenhancement input or output device(s) included within the lightingdevice.

If provided on the system 10, additional system elements, such as astandalone sensor or an additional user interface device, they may besimilarly implemented using an architecture like one of the devices ofFIGS. 7-9. For example, an additional other user interface device (UI)might utilize an arrangement similar to the mobile device of FIG. 9,albeit possibly with only one transceiver compatible with the networkingtechnology of the particular premises (e.g. to reduce costs).

For information about other examples of intelligent lighting devices,which may be suitable for use in a networked lighting system like thatof FIG. 1A, attention may be directed to U.S. application Ser. No.13/463,594 Filed May 3, 2012 entitled “LIGHTING DEVICES WITH INTEGRALSENSORS FOR DETECTING ONE OR MORE EXTERNAL CONDITIONS AND NETWORKEDSYSTEM USING SUCH DEVICES,” the disclosure of which is entirelyincorporated herein by reference.

As also outlined above, aspects of the interactive user interface andany associated distributed processing techniques of the lighting devices11 may be embodied in programming of the appropriate system elements,particularly for the processors of intelligent lighting devices 11.Program aspects of the technology discussed above therefore may bethought of as “products” or “articles of manufacture” typically in theform of executable code and/or associated data (software or firmware)that is carried on or embodied in a type of machine readable medium.“Storage” type media include any or all of the tangible memory of thecomputers, processors or the like, or associated modules thereof, suchas various semiconductor memories, tape drives, disk drives and thelike, which may provide non-transitory storage at any time for thesoftware or firmware programming. All or portions of the programming mayat times be communicated through the Internet or various othertelecommunication networks. Such communications, for example, may enableloading of the software from one computer or processor into another, forexample, from a management server or host computer of the lightingsystem service provider (e.g. implemented like the server computer shownat 27) into any of the lighting devices, etc. of or coupled to thesystem 10 at the premises 21, including programming for individualelement functions, programming for user interface functions andprogramming for distributed processing functions. Thus, another type ofmedia that may bear the software/firmware program elements includesoptical, electrical and electromagnetic waves, such as used acrossphysical interfaces between local devices, through wired and opticallandline networks and over various air-links. The physical elements thatcarry such waves, such as wired or wireless links, optical links or thelike, also may be considered as media bearing the software. As usedherein, unless restricted to non-transitory, tangible “storage” media,terms such as computer or machine “readable medium” refer to any mediumthat participates in providing instructions to a processor forexecution.

It will be understood that the terms and expressions used herein havethe ordinary meaning as is accorded to such terms and expressions withrespect to their corresponding respective areas of inquiry and studyexcept where specific meanings have otherwise been set forth herein.Relational terms such as first and second and the like may be usedsolely to distinguish one entity or action from another withoutnecessarily requiring or implying any actual such relationship or orderbetween such entities or actions. The terms “comprises,” “comprising,”“includes,” “including,” or any other variation thereof, are intended tocover a non-exclusive inclusion, such that a process, method, article,or apparatus that comprises a list of elements does not include onlythose elements but may include other elements not expressly listed orinherent to such process, method, article, or apparatus. An elementproceeded by “a” or “an” does not, without further constraints, precludethe existence of additional identical elements in the process, method,article, or apparatus that comprises the element.

Unless otherwise stated, any and all measurements, values, ratings,positions, magnitudes, sizes, and other specifications that are setforth in this specification, including in the claims that follow, areapproximate, not exact. They are intended to have a reasonable rangethat is consistent with the functions to which they relate and with whatis customary in the art to which they pertain.

While the foregoing has described what are considered to be the bestmode and/or other examples, it is understood that various modificationsmay be made therein and that the subject matter disclosed herein may beimplemented in various forms and examples, and that they may be appliedin numerous applications, only some of which have been described herein.It is intended by the following claims to claim any and allmodifications and variations that fall within the true scope of thepresent concepts.

What is claimed is:
 1. A system, comprising: lighting devices, eachrespective lighting device comprising: an illumination light source forillumination of a space; a communication interface configured to enablecommunication via a network link; and a processor coupled to control thelight source and coupled to communicate via the communication interfaceand the network link with one or more others of the lighting devices andconfigured to control operations of at least the respective lightingdevice; wherein: at least one of the lighting devices includes a userinput sensor configured to detect user inputs of a user, wherein theuser inputs include an audio input and the user input sensor includes amicrophone to receive the audio input; one or more of the processors ofthe lighting devices is further configured to: digitize, the audio inputdetected via the microphone, into an audio data segment; convert theaudio data segment into a parsed representation of speech; recognize acommand in the parsed representation of speech by: sending, to a server,the parsed representation of speech and receiving, from the server, therecognized command in response to sending the parsed representation ofspeech; or analyzing, on one or more of the lighting devices, the parsedrepresentation of speech to recognize the command in the parsedrepresentation of speech; control an operation of at least one of thelighting devices as a function of the recognized lighting command. 2.The system of claim 1, further comprising a data communication networkto provide data communications amongst the lighting devices, wherein:the data communication network is configured to further provide datacommunications for at least one of the lighting devices via a wide areanetwork with the server.
 3. The system of claim 1, wherein: at least oneof the lighting devices includes an output component configured toprovide information output to the user in an audio or a visual format;the output component includes at least one of a display, a projector, aspeaker, or a visual output light source; and one or more of theprocessors of the lighting devices is further configured to control theinformation output to the user in the audio or the visual format by theoutput component in response to the detected user inputs.
 4. The systemof claim 3, wherein: the output component includes the visual outputlight source; and the one or more of the processors of the lightingdevices is further configured to control the information output to theuser in the visual format by changing light output of the visual outputlight source in response to the detected user inputs.
 5. The system ofclaim 4, wherein the changed light output provides a visual cue to theuser.
 6. The system of claim 5, wherein the changed light outputproduces a repeating pattern or change in a lighting related parameter.7. The system of claim 1, wherein: one of the lighting devices includesboth the user input sensor and the output component.
 8. The system ofclaim 1, wherein: the user input sensor is further configured to detectan optical input; and the one or more of the processors of the lightingdevices is further configured to: process the detected optical input toidentify a gesture of a user; and control the operation of at least oneof the lighting devices as a function of the identified gesture of theuser.
 9. The system of claim 8, wherein the user input sensor furtherincludes an image detector.
 10. The system of claim 8, wherein the userinput sensor further includes a device configured for detection of adirection and intensity of a received light.
 11. A lighting devicecomprising: a light source; a communication interface configured toenable communication via a network link; a user input sensor configuredto detect a user input of a user, wherein the user input includes anaudio input and the user input sensor includes a microphone to receivethe audio input; and a processor coupled to control the light source andcoupled to communicate via the communication interface, wherein theprocessor is configured to respond to the user input detected by theuser input sensor by: digitizing, the audio input detected via themicrophone, into an audio data segment; converting the audio datasegment into a parsed representation of speech; and recognizing acommand in the parsed representation of speech by: sending, to a server,the parsed representation of speech and receiving, from the server, therecognized command in response to sending the parsed representation ofspeech; or analyzing, on the lighting device, the parsed representationof speech to recognize the command in the parsed representation ofspeech.
 12. The lighting device of claim 11, further comprising: anoutput component configured to provide information output to the user inan audio or a visual format; and wherein: the output component includesat least one of a display, a projector, a speaker, or the light source,and the processor is further configured to control the informationoutput to the user in the audio or the visual format by the outputcomponent in response to the detected user input.
 13. The lightingdevice of claim 12, wherein: the output component includes the lightsource; and the processor is further configured to control theinformation output to the user in the visual format by changing lightoutput of the light source in response to the detected user input. 14.The lighting device of claim 13, wherein the changed light outputprovides a visual cue to the user.
 15. The lighting device of claim 14,wherein the changed light output produces a repeating pattern or changein a lighting related parameter.
 16. The lighting device of claim 11,wherein: the user input sensor is further configured to detect anoptical input; and the processor is further configured to: process thedetected optical input to identify a gesture of a user; and control anoperation of the lighting device as a function of the identified gestureof the user.
 17. The lighting device of claim 16, wherein the user inputsensor further includes an image detector.
 18. The lighting device ofclaim 16, wherein the user input sensor further includes a deviceconfigured for detection of a direction and an intensity of receivedlight.
 19. The lighting device of claim 11, wherein the parsedrepresentation of speech includes at least one word.
 20. A devicecomprising: a communication interface configured to enable communicationvia a network link; a microphone; a speaker; a visual output lightsource; and circuitry, including a processor coupled to control thevisual output light source and coupled to communicate via thecommunication interface, the circuitry being configured to: digitize, anaudio input detected via the microphone, into an audio data segment;convert the audio data segment into a parsed representation of speech;recognize a command in the parsed representation of speech by: sending,to a server, the parsed representation of speech and receiving, from theserver, the recognized command in response to sending the parsedrepresentation of speech; or analyzing, on the lighting device, theparsed representation of speech to recognize the command in the parsedrepresentation of speech; and respond to the recognized command,including to provide a corresponding audio output via the speaker andprovide a visual response via the visual output light source.